Daisy documentation
Book Index

Daisy documentation

Table of Contents

1 Documentation Home

These pages contain the Daisy documentation of the Daisy 1.5 release. The main Daisy site can be found here.

The documentation is also available published as a Daisy-book.

For an end-user introduction to Daisy, have a look at the video tutorials.

2 Installation

2.1 Downloading Daisy

Packaged versions of Daisy can be found in the distribution area (Sourceforge). This includes everything required to run Daisy, except for:

If you don't have these already, the installation of these will be covered further on.

Consider subscribing to the Daisy mailing list to ask questions and talk with fellow Daisy users and developers.

There is also information available about the source code.

2.2 Installation Overview

Daisy is a multi-tier application, consisting of a repository server and a publication layer. Next to those, a database server (MySQL) is required. All together, this means three processes, which can run on the same server or on different servers.

The Daisy binary distribution packs most of the needed software together, the only additional things you'll need is a Java Virtual Machine for your platform, and MySQL. All libraries and applications shipped with Daisy are the original, unmodified distributions that will be configured as part of the installation. We've only grouped them in one download for your convenience.

If you follow the instructions in this document, you can have Daisy up and running in less than an hour.

The diagram below gives an overview of the the setup. All shown port numbers are configurable of course.

2.2.1 Platform Requirements

We have tested the Daisy installation on Windows 2000/XP, GNU/Linux and MacOSX. Other unixes like Solaris should also work, though we don't test that ourselves.

2.2.2 Memory Requirements

By default, the Daisy Wiki and Daisy Repository Server are started with a maximum heap size of 128 MB each. To this you need to add some overhead of the JVMs themselves, and then some memory for MySQL, the OS and its (filesystem) caches. This doesn't mean all this memory will be used, that will depend on usage intensity.

2.2.3 Required knowledge

These installation instructions assume you're comfortable with installing software, editing configuration (XML) files, running applications from the command line, setting environment variables, and that sort of stuff.

2.2.4 Can I use Oracle, PostgreSQL, MS-SQL, ... instead of MySQL? Websphere, Weblogic, Tomcat, ... instead of Jetty? What is this Merlin thing?

Daisy contains the necessary abstractions to support different database engines, though we currently only support MySQL. Users are welcome to contribute and maintain different databases (ask on the mailing list how to get started).

The Daisy Wiki webapp should be able to run in any servlet container (at least one that can run unpacked webapps, and as far as there aren't any Cocoon-specific issues), but we ship Jetty by default. For example, using Tomcat instead of Jetty is very simple and is described on this page.

The Daisy Repository Server runs on top of a component runtime platform called Merlin. Generally you won't be aware of this, but if you see "merlin" popping up in process names, that means it is basically the same as the Daisy Repository Server.

2.3 Installing a Java Virtual Machine

Daisy requires either the Java SDK 1.4.2 or the Java JDK 5.0.

You can download the Java JDK 5.0 from here on the Sun site (take the JDK, not the JRE). Install it now if you don't have it already.

After installation, make sure the JAVA_HOME environment variable is defined and points to the correct location (i.e., the directory where Java is installed). To verify this, open a command prompt or shell and enter:

For Windows:
%JAVA_HOME%/bin/java -version

For Linux:
$JAVA_HOME/bin/java -version

This should print out something like:

java version "1.4.2_xx"

or

java version "1.5.0"

2.3.1 Installing JAI (Java Advanced Imaging) -- optional

If you want images (especially PNG) to appear in PDFs, it is highly advisable to install JAI, which you can download from the JAI homepage. Just take the "JDK Install" option, this will make JAI support globally available.

2.4 Installing MySQL

Daisy requires one of the following MySQL versions:

MySQL can be downloaded from mysql.com. Install it now, and start it (often done automatically by the install).

Windows users can take the "Windows Essentials" package. During installation and the configuration wizard, you can leave most things to their defaults. In particular, be sure to leave the "Database Usage" to "Multifunctional Database", and leave the TCP/IP Networking enabled (on port 3306). When it asks for Windows options, check the option "Include Bin Directory In Windows Path".

Linux users: install the "MySQL server" and "MySQL client" packages. Installing the MySQL server RPM will automatically initialize and start the MySQL server.

2.4.1 Creating MySQL databases and users

MySQL is used by both the Daisy Repository Server and JMS (ActiveMQ). Therefore, we are now going to create two databases and two users.

Open a command prompt, and start the MySQL client as root user:

mysql -uroot -pYourRootPassword

On some systems, the root user has no password, in which case you can drop the -p parameter.

Now create the necessary databases, users and access rights by entering (or copy-paste) the commands below in the mysql client. What follows behind the IDENTIFIED BY is the password for the user, which you can change if you wish. The daisy@localhost entries are necessary because otherwise the default access rights for anonymous users @localhost will take precedence. If you'll run MySQL on the same machine as the Daisy Repository Server, you only need the @localhost entries.

CREATE DATABASE daisyrepository CHARACTER SET 'utf8';
GRANT ALL ON daisyrepository.* TO daisy@'%' IDENTIFIED BY 'daisy';
GRANT ALL ON daisyrepository.* TO daisy@localhost IDENTIFIED BY 'daisy';
CREATE DATABASE activemq CHARACTER SET 'utf8';
GRANT ALL ON activemq.* TO activemq@'%' IDENTIFIED BY 'activemq';
GRANT ALL ON activemq.* TO activemq@localhost IDENTIFIED BY 'activemq';

2.5 Extract the Daisy download

Extract the Daisy download. On Linux/Unix you can extract the .tar.gz file as follows:

tar xvzf daisy-<version>.tar.gz

On non-Linux unixes (Solaris notably), use the GNU tar version if you experience problems extracting.

On Windows, use the .zip download, which you can extract using a tool like WinZip.

After extraction, you will get a directory called daisy-<version>. This directory is what we will call from now on the DAISY_HOME directory. You may set a global environment variable pointing to that location, or you can do it each time in the command prompt when needed.

2.6 Daisy Repository Server

2.6.1 Initialising and configuring the Daisy Repository

Open a command prompt or shell and set an environment variable DAISY_HOME, pointing to the directory where Daisy is installed.

Windows:
set DAISY_HOME=c:\daisy-1.5

Linux:
export DAISY_HOME=/home/daisy_user/daisy-1.5

Then go to the directory <DAISY_HOME>/install, and execute: 

daisy-repository-init

Follow the instructions on screen. The installation will (1) initialise the database tables for the repository server and (2) create a Daisy data directory containing customized configuration files.

Make sure that name of the path that you provide for the Daisy data directory does not contain spaces, otherwise the startup of the repository server will fail.

2.6.2 Starting the Daisy Repository Server

Still in the same command prompt (or in a new one, but make sure DAISY_HOME is set), go to the directory <DAISY_HOME>/repository-server/bin, and execute:

daisy-repository-server <location-of-daisy-data-dir>

In which you replace <location-of-daisy-data-dir> with the location of the daisy data directory created in the previous step.

The start-up can take a few seconds, though nothing special is printed to the screen to indicate it is completely started (the prompt will not return).

2.7 Daisy Wiki

2.7.1 Initialising the Daisy Wiki

Before you can run the Daisy Wiki, the repository needs to be initialised with some document types, a "guest" user, a default ACL configuration, etc.

Open a command prompt or shell, make sure DAISY_HOME is set, go to the directory <DAISY_HOME>/install, and execute:

daisy-wiki-init

The program will start by asking a login and password, enter here the user created during the execution of daisy-repository-init (the default was testuser/testuser). It will also ask for the URL where the repository is listening, you can simply press enter here.

If everything goes according to plan, the program will now print out some informational messages and end with "Finished.".

2.7.2 Creating a "wikidata" directory

Similar to the data directory of the Daisy repository server, the Daisy Wiki also has its own data directory (which we call the "wikidata directory").

To set up this directory, open a command prompt or shell, make sure DAISY_HOME is set, go to the directory <DAISY_HOME>/install, and execute:

daisy-wikidata-init

and follow the instructions on-screen.

Since the Daisy Wiki and the Daisy repository server are two separate applications (which might be deployed on different servers), each has its own data directory.

2.7.3 Creating a Daisy Wiki Site

The Daisy Wiki has the concept of multiple sites, these are multiple views on top of the same repository. You need at least one site to do something useful with the Daisy Wiki, so we are now going to create one.

Open a command prompt or shell, make sure DAISY_HOME is set, go to the directory <DAISY_HOME>/install, and execute:

daisy-wiki-add-site <location of wikidata directory>

The application starts by asking the same parameters as for daisy-wiki-init.

Then it will ask a name for the site. This should be a name without spaces. If you're inspirationless, enter something like "test" or "main".

Then it will ask for the sites directory location, for which the presented default should be OK, so just press enter.

2.7.4 Starting the Daisy Wiki

Open a command prompt or shell and make sure DAISY_HOME is set.

Go to the directory <DAISY_HOME>/daisywiki/bin, and execute: 

daisy-wiki <location of wikidata directory>

Background info: this will start Jetty (a servlet container) with the webapp found in <DAISY_HOME>/daisywiki/webapp.

2.8 Finished!

Now you can point your web browser to:

http://localhost:8888/daisy/

(Note the final slash on the end)

To be able to create or edit documents, you will have to change the login, you can use the user you created for yourself while running daisy-repository-init (the default was testuser/testuser).

To start the Daisy repository server and Daisy Wiki after the initial installation, see the summary here, or even better, set up service (init) scripts to easily/automatically start and stop Daisy.

2.9 1.4 to 1.5-M1 upgrade

Daisy 1.5-M1 was released on March 22, 2006.

2.9.1 Changes

2.9.1.1 Features

2.9.1.2 Non-features

2.9.1.3 Bug fixes

2.9.2 Compatibility

2.9.2.1 Document type specific stylesheet

There is small difference in the output of the publisher requests which influences the document type specific stylesheets. Don't worry, it is a very minor change, depending on how your stylesheets are written they might even require no change at all.

Where the input previously contained this element hierarchy:

document > d:document

it has now become:

document > p:publisherResponse > d:document

Where "p:" is the publisher namespace, thus bound as:

xmlns:p="http://outerx.org/daisy/1.0#publisher"

If you want to have stylesheets that work with both releases, you can write them so that they first test for the presence of document/d:document and if not available use document/p:publisherResponse/d:document

2.9.2.2 Custom styling: attributes on links

The links in document content were previously annotated with the attributes daisyDocumentName, daisyFileName and daisyNavigationPath. These have been removed in favour of a more powerful system, as explained in this mail.

If you have custom document type specific XSLs or book publication types, you might want to do a search for @daisyDocumentName used in XPath expressions, and replace it by p:linkInfo/@documentName, the p prefix should be declared as:

xmlns:p="http://outerx.org/daisy/1.0#publisher"

2.9.2.3 Auto-redirecting between sites

The Daisy Wiki now has the ability to automatically redirect to another site if a document is better suited to be displayed over there. This is enabled by default, and can be disabled by adding the following element in the skinconf.xml of each site for which you want to disable this:

<siteSwitching mode="stay"/>

2.9.2.4 Field type in HTTP interface

A small change has been done to the field type XML. If you have any programs making direct use of the HTTP interface, you'll need to adjust them.

Where you previously had a <d:selectionList> element for specifying a static selection list, the <d:selectionList> element now contains an extra <d:staticSelectionList> element which then contains the elements previously contained in <d:selectionList>. This change was done to allow for alternative selection list implementations.

2.9.2.5 Publisher XML Beans generated classes

Applies when using custom publisher requests build up via the Java objects (instead of written in XML).

The structure of the XML Schema for the publisher requests has changed a little bit, which has caused some small changes in the nesting of the classes generated by XMLBeans for this XML Schema. Adjusting your code for this is straightforward.

2.9.3 Upgrading

2.9.3.1 Before starting

Shutdown Daisy (the Repository Server, the Daisy Wiki, and the OpenJMS server)

Make backups! More specifically:

daisy-backup-tool -b -d $DAISYDATA_DIR -l $DAISY_BACKUP_DIR -o $DAISY_HOME/openjms -a additional-entries.xml

The '-a additional-entries.xml' is not required to make a backup, but if specified you can backup daisywiki specific files.  If you want to backup these files you must first create the addition-entries.xml file, an example can be found in the backup documentation.

2.9.3.2 Upgrade blobstore to hierarchical format

The blobstore structure has changed from being flat to being hierarchical, therefore a conversion must be made.  Luckily there is a tool that will convert your existing data to the correct format.  It can be executed by running the following command :

<DAISY_HOME>/bin/daisy-blobstore-convertor <path-to-blobstore>

This should be quite a painless procedure but seeing the delicate nature of this operation it might be wise to create a backup first.

2.9.3.3 Run database upgrade script

cd <DAISY_HOME>/misc
mysql -Ddaisyrepository -udaisy -p<password>
[then on the mysql prompt]
\. daisy-1_4-to-1_5_M1.sql

2.9.3.4 Create pubreqs directory and update myconfig.xml

In the daisy data directory, create a new empty subdirectory called pubreqs (all lowercase). This directory becomes thus a sibling of the blobstore, indexstore, conf and logs directories.

Then open the following file in a text editor:

<daisy data dir>/conf/myconfig.xml

And add the following as a child of the root <targets> element (the order of this and the other <target> elements does not matter) 

  <target path="/daisy/extensions/publisher/publisher">
    <configuration>
      <repositoryUser login="internal" password="defaultpwd"/>
      <publisherRequestDirectory>/somewhere/pubreqs</publisherRequestDirectory>
    </configuration>
  </target>

Change the value of the password attribute to the same value as the password attribute on the other <repositoryUser> elements in the same file.

Change the content of the <publisherRequestDirectory> to contain the absolute path to the new pubreqs directory you just created.

2.9.3.5 Add headless option for repository server

If you are using the service wrapper or another custom startup script, you should add the java.awt.headless parameter, e.g. in dsy_repo_wrapper.conf the line:

wrapper.java.additional.4=-Djava.awt.headless=true

If you don't do this, image thumbnail generation will fail.

2.9.3.6 Update classpath for Jetty 5

If you are using the service wrapper or another custom startup script for the Daisy Wiki, note that Daisy now ships with Jetty 5. For the wrapper, this means the classpath has changed. See the updated dsy_wiki_wrapper.conf file.

2.9.3.7 Copy over the old configuration

Edit <DAISY_HOME>/openjms/bin/setenv.(sh|bat), uncomment the line defining the CLASSPATH and put the MySQL driver in the CLASSPATH, which can be found at (substitute <DAISY_HOME> by its actual location):

<DAISY_HOME>/lib/mysql/jars/mysql-connector-java-3.1.7-bin.jar

2.9.3.8 Start the servers

Start OpenJMS, the Daisy Repository Server, and the Daisy Wiki.

If necessary, first update the DAISY_HOME and OPENJMS_HOME variables to point to the location of the new Daisy version.

2.9.3.9 Upgrade Daisy Wiki schema

Run the daisy-wiki-init script to update the Daisy Wiki schema types:

cd <DAISY_HOME>/install
daisy-wiki-init

Run daisy-wiki-init (for ImageThumbnail etc.)

2.9.3.10 [optional] Image thumbnail generation

Daisy 1.5 can generate image thumbnails and extract their width and height to store them in fields of the Image document type. If you want to do this for all your current images, you can do so by running a document task. For this, first make sure you're logged in as Administrator, then go to the Document Tasks screen (the link is in the user menu) and create a new document task. Select the documents using the query:

select name where documentType = 'Image'

You might first want to try this with one document to see if everything is OK.

Then go to the next screen, and use the following script:

var document = repository.getDocument(variantKey, true);
document.setField("ImageWidth", new java.lang.Long(0));
document.save();

This script "touches" the document by setting a field to a dummy value. Otherwise, the document wouldn't be changed and no save will be performed.

2.10 1.5-M1 to 1.5-M2 upgrade

Daisy 1.5 Milestone 2 was released on June 21, 2006.

2.10.1 Changes

2.10.1.1 Important changes: read this section!

2.10.1.2 Other new features

2.10.1.3 Bug fixes & minor improvements

2.10.2 Compatibility notes

2.10.2.1 Skins

[todo: add helpful notes here on updating custom skins based on the old default skin]

2.10.2.2 Custom book publication types

If you made custom book publication types or document type specific XSLs for use in books, they might need some small changes due to the introduction of the wikidata directory.

In the query-styling.xsl of the publication type, if you want to import the default query styling XSL, the include should now be:

<xsl:include href="wikidata:books/publicationtypes/common/default-query-styling.xsl"/>

If in the custom document type specific XSL you included the default book-document-to-html.xsl file, the import should now be:

<xsl:include href="wikidata:books/publicationtypes/common/book-document-to-html.xsl"/>

2.10.2.3 Databases

MySQL 4.0 is not supported anymore. You might still be able to get it to work, but it wasn't worth the effort anymore for us to support it in the default configuration.

2.10.2.4 JmsClient API

If you were using the remote Java API and constructing a JmsClientImpl to enable remote cache invalidation through JMS, you'll notice the constructor has changed: it takes a new parameter clientID (the JMS client ID, which is a string of your choice, but should be different for each client), and the topicConnectionFactory and queueConnectionFactory parameters have merged into just one connectionFactory argument (since we're now using JMS 1.1 APIs internally, as a result of the switch to ActiveMQ).

2.10.3 Upgrading

Important: these are instructions for upgrading from Daisy 1.5-M1. To upgrade from older releases, e.g. Daisy 1.4, please follow the aggregated instructions for upgrading from 1.4 to 1.5-final, which can be found here.

2.10.3.1 Download and extract installation files

  1. Rename your existing <DAISY_HOME> directory. During these instructions, we will refer to this location as <OLD_DAISY_HOME>.
  2. Download the Daisy 1.5 installation files from here.
  3. Extract the installation archive. Make sure that  your <DAISY_HOME> variable points to that newly created directory, if necessary, move or rename that directory (or adapt your <DAISY_HOME> environment variable).

2.10.3.2 Switching from OpenJMS to ActiveMQ

2.10.3.2.1 Make sure no messages are left in OpenJMS

Before moving to ActiveMQ, it is a good idea to check if there are no messages waiting to be processed in OpenJMS's queues. Usually JMS messages in Daisy are processed quickly, so unless you have had high activity right before shutting down the Daisy repository, there will normally be no more messages waiting to be processed.

To be sure, you can do a quick check like this:

If there would still be messages, just start the repostory server again, and wait till they have all been processed.

2.10.3.2.2 Creating a database for ActiveMQ

Start the MySQL client:

mysql -uroot

and execute:

CREATE DATABASE activemq;
GRANT ALL ON activemq.* TO activemq@'%' IDENTIFIED BY 'activemq';
GRANT ALL ON activemq.* TO activemq@localhost IDENTIFIED BY 'activemq';

ActiveMQ will automatically create its database tables the first time the repository server is launched.

2.10.3.2.3 ActiveMQ configuration

Create the ActiveMQ configuration :

cp <DAISY_HOME>/repository-server/conf/activemq-conf.xml.template <DAISY_DATA>/conf/activemq-conf.xml
cp <DAISY_HOME>/repository-server/conf/login.config <DAISY_DATA>/conf/
cp <DAISY_HOME>/repository-server/conf/users.properties <DAISY_DATA>/conf/
cp <DAISY_HOME>/repository-server/conf/groups.properties <DAISY_DATA>/conf/

Update myconfig.xml (<DAISY_DATA>/conf/myconfig.xml):

Look for the following element:

<target path="/daisy/jmsclient/jmsclient">

and replace the <configuration> element inside it with:

<configuration>
  <jmsConnection>
    <clientId>daisy-repository</clientId>
    <credentials username="admin" password="jmsadmin"/>
    <initialContext>
      <property name="java.naming.provider.url" value="vm://DaisyJMS?brokerConfig=xbean:file:${daisy.datadir}/conf/activemq-conf.xml"/>
      <property name="java.naming.factory.initial" value="org.apache.activemq.jndi.ActiveMQInitialContextFactory"/>
      <property name="queue.fullTextIndexerJobs" value="fullTextIndexerJobs"/>
      <property name="topic.daisy" value="daisy"/>
    </initialContext>
    <connectionFactoryName>ConnectionFactory</connectionFactoryName>
  </jmsConnection>
</configuration>

If you are using MySQL 5 (thus not 4.1), then you need to edit <DAISY_DATA>/conf/activemq-conf.xml. Look for the following line (here split on two lines for readability):

<property name="url" value="jdbc:mysql://localhost/activemq?relaxAutoCommit=true&amp;
                         useServerPrepStmts=false&amp;sessionVariables=storage_engine=InnoDB"/>

In this, you should remove the "useServerPrepStmts=false&amp;", so that it becomes:

<property name="url" value="jdbc:mysql://localhost/activemq
          ?relaxAutoCommit=true&amp;sessionVariables=storage_engine=InnoDB"/>

2.10.3.3 Configuring the Driver Registrar component

This new component registers JDBC drivers that are used by datasources in the repository-server.

Background info: previously, the JDBC drivers were configured as part of the datasource, but since JDBC connections are now needed by multiple datasources in the same VM (both the repository server and ActiveMQ), their registration is now moved into a specific component.

Edit <DAISY_DATA>/conf/myconfig.xml, and add a new target :

<target path="/daisy/driverregistrar/driverregistrar">
  <configuration>
    <drivers>
      <driver>
        <classpath>${daisy.home}/lib/mysql/jars/mysql-connector-java-3.1.12-bin.jar</classpath>
      	<classname>com.mysql.jdbc.Driver</classname>
      </driver>
    </drivers>
  </configuration>
</target>

Notice the upgrade from mysql-connector 3.1.7 to mysql-connector 3.1.12

You may now also remove the references to JDBC drives from the datasource component.  In the same file look for "/daisy/datasource/datasource".  In the configuration of this component look for the "driverClasspath" and "driverClassName" elements.  You may simply remove both these elements like this :

<target path="/daisy/datasource/datasource">
    <configuration>
      <username>daisy</username>
      <password>daisy</password>
      <url>jdbc:mysql://localhost/daisyrepository?useServerPrepStmts=false</url>
      <driverClasspath>REMOVE THIS LINE</driverClasspath>
      <driverClassName>REMOVE THIS LINE</driverClassName>
      <maxActive>20</maxActive>
      <maxIdle>8</maxIdle>
      <minIdle>0</minIdle>
      <maxWait>5000</maxWait>
    </configuration>
  </target>

2.10.3.4 Updating wrapper configuration for the Repository Server

If you are using the service wrapper (or custom startup scripts) to start the repository server, then you will need to upgrade the configuration/scripts to pass some additional parameters. In dsy_repo_wrapper.conf, these have to be added (see also the full script): 

wrapper.java.additional.5=-Dorg.apache.commons.logging.Log=org.outerj.daisy.logging.DaisyLog
wrapper.java.additional.6=-Djava.security.auth.login.config=%DAISY_DATADIR%/conf/login.config
wrapper.java.additional.7=-Ddaisy.datadir=%DAISY_DATADIR%
wrapper.java.additional.8=-Ddaisy.home=%DAISY_HOME%

2.10.3.5 Optional: remove absolute paths in myconfig.xml

Daisy now allows to use properties in the myconfig.xml to avoid absolute paths. If you want, you can update your myconfig.xml and change absolute references to the daisy data directory and daisy home directory by ${daisy.datadir} and ${daisy.home} respectively. Doing this will allow to move your data directory around without changing the myconfig.xml.

2.10.4 Start repository server

At this point, start the repository server, as this is needed for the remainder of the upgrade.

It is not needed anymore to start OpenJMS.

2.10.5 Create wikidata directory

From now on, configurations parameters, skins, doctype XSLs, the book publication types and book store, the wiki log files, in short all 'wiki-instance-specific' stuff is stored in a separate "wikidata directory" (not to be confused with the daisydata directory, which performs a similar role but then for the repository server).

We will now create such a directory and copy your existing configuration to it.

To create the wikidata directory, go to <DAISY_HOME>/install and execute:

daisy-wikidata-init

The program will ask you for a location for this directory. Just choose some location for this, for example put it next to (but not inside) the repository daisydata directory.

The program will also ask for the location of the repository daisydata directory, as it will take some configuration values from there (internal & jms user password) and put them in the wiki config.

It will also try to create the registrar user, but since you're doing an upgrade that user will already exist so it will simply skip that. However, you will need to enter the password for the registrar user. This can be found in <OLD_DAISY_HOME>/daisywiki/webapp/WEB-INF/cocoon.xconf, and look there for something like:

<registrarUser login="registrar" password="b6f84c35249a08d11ad6fa8279e2f9"/>

Just copy-and-paste that password to the input prompt.

Once the daisy-wikidata-init script has ended, we can copy over the remaining things:

If you made any configuration changes to the repository manager, JMS client or registrar component which were previously in the cocoon.xconf, then you can now find these in <wikidata dir>/daisy.xconf.

2.10.5.1 Updating wrapper configuration for the Daisy Wiki

Similarly as in the previous section, the Daisy Wiki needs the following additional parameter in dsy_wiki_wrapper.conf (see also the full script):

wrapper.java.additional.7=-Ddaisywiki.data=%DAISYWIKI_DATADIR%

The DAISYWIKI_DATADIR environment variable has to be declared in the shell script (dsy_wiki.sh or equivalent):

#! /bin/sh

DAISY_HOME=/home/daisy/daisy ; export DAISY_HOME
JETTY_HOME=$DAISY_HOME/daisywiki/jetty ; export JETTY_HOME
DAISYWIKI_DATADIR=/path/to/your/daisywikidata ; export DAISYWIKI_DATADIR

[.....]

2.10.6 Start the Daisy Wiki

Now you can also start the Daisy Wiki.

2.10.7 After-upgrade notes

2.10.7.1 OpenJMS cleanup

Since OpenJMS is not used anymore, some cleanup you might want to to:

2.10.7.2 ActiveMQ authentication

With the procedure we followed above, ActiveMQ still uses default passwords. If the ActiveMQ port is publicly accessible, it is of course very much recommended to change these. Here is how. The password needs to be changed in the following locations:

Original setting:

In  this file:
<datadir>/conf/users.properties

in this line:
admin=jmsadmin

For the connection made by the Repository Server:

In this file:
<datadir>/conf/myconfig.xml

in this line:
<credentials username="admin" password="jmsadmin"/>

For the connection made by the Daisy Wiki:

In this file:
<wikidata dir>/daisy.xconf

in this line:
<credentials username="admin" password="jmsadmin"/>

After making these changes, stop both the Daisy Wiki and the Repository Server, and then restart them (first the repository, then the wiki).

2.11 1.5-M2 to 1.5 final upgrade

2.11.1 Changes

Only some small changes and bug fixes in this release:

2.11.2 Upgrading

Nothing special needs to be done to upgrade from Daisy 1.5-M2 to this release.

Just stop the repository and wiki servers, extract the new release, adjust DAISY_HOME or move the old Daisy install out of the way, and restart the servers.

2.12 1.4 to 1.5 upgrade

Daisy 1.5 was released on August 16, 2006.

Read this document carefully, especially the upgrading instructions. Take your time to do the upgrade. This release contains various changes that will make future updates a lot easier.

2.12.1 Changes

Some important changes that affect deployment or compatibility: READ THIS SECTION

Other changes:

To also see bug fixes or some non-functional changes, check the change lists for each individual 1.5 Milestone release.

2.12.2 Compatibility

2.12.2.1 Document type specific stylesheet

There is small difference in the output of the publisher requests which influences the document type specific stylesheets. Don't worry, it is a very minor change, depending on how your stylesheets are written they might even require no change at all.

Where the input previously contained this element hierarchy:

document > d:document

it has now become:

document > p:publisherResponse > d:document

Where "p:" is the publisher namespace, thus bound as:

xmlns:p="http://outerx.org/daisy/1.0#publisher"

If you want to have stylesheets that work with both releases, you can write them so that they first test for the presence of document/d:document and if not available use document/p:publisherResponse/d:document

2.12.2.2 Custom styling: attributes on links

The links in document content were previously annotated with the attributes daisyDocumentName, daisyFileName and daisyNavigationPath. These have been removed in favour of a more powerful system, as explained in this mail.

If you have custom document type specific XSLs or book publication types, you might want to do a search for @daisyDocumentName used in XPath expressions, and replace it by p:linkInfo/@documentName, the p prefix should be declared as:

xmlns:p="http://outerx.org/daisy/1.0#publisher"

2.12.2.3 Auto-redirecting between sites

The Daisy Wiki now has the ability to automatically redirect to another site if a document is better suited to be displayed over there. This is enabled by default, and can be disabled by adding the following element in the siteconf.xml of each site for which you want to disable this:

<siteSwitching mode="stay"/>

2.12.2.4 Field type in HTTP interface

A small change has been done to the field type XML. If you have any programs making direct use of the HTTP interface, you'll need to adjust them.

Where you previously had a <d:selectionList> element for specifying a static selection list, the <d:selectionList> element now contains an extra <d:staticSelectionList> element which then contains the elements previously contained in <d:selectionList>. This change was done to allow for alternative selection list implementations.

2.12.2.5 Publisher XML Beans generated classes

Applies when using custom publisher requests build up via the Java objects (instead of written in XML).

The structure of the XML Schema for the publisher requests has changed a little bit, which has caused some small changes in the nesting of the classes generated by XMLBeans for this XML Schema. Adjusting your code for this is straightforward.

2.12.2.6 Skins

[todo: add helpful notes here on updating custom skins based on the old default skin]

2.12.2.7 Custom book publication types

If you made custom book publication types or document type specific XSLs for use in books, they might need some small changes due to the introduction of the wikidata directory.

In the query-styling.xsl of the publication type, if you want to import the default query styling XSL, the include should now be:

<xsl:include href="wikidata:books/publicationtypes/common/default-query-styling.xsl"/>

If in the custom document type specific XSL you included the default book-document-to-html.xsl file, the import should now be:

<xsl:include href="wikidata:books/publicationtypes/common/book-document-to-html.xsl"/>

2.12.2.8 Databases

MySQL 4.0 is not supported anymore. You might still be able to get it to work, but it wasn't worth the effort anymore for us to support it in the default configuration.

2.12.2.9 JmsClient API

If you were using the remote Java API and constructing a JmsClientImpl to enable remote cache invalidation through JMS, you'll notice the constructor has changed: it takes a new parameter clientID (the JMS client ID, which is a string of your choice, but should be different for each client), and the topicConnectionFactory and queueConnectionFactory parameters have merged into just one connectionFactory argument (since we're now using JMS 1.1 APIs internally, as a result of the switch to ActiveMQ).

2.12.3 Upgrading

Important: these are instructions for upgrading from Daisy 1.4 (or 1.4.1). To upgrade from older releases, e.g. Daisy 1.3, you first need to follow the instructions to upgrade to 1.4, and then these instructions (you don't need to actually download or install 1.4, just follow the instructions). To upgrade from Daisy 1.5-M1, follow the instructions for upgrading to 1.5-M2 (from 1.5-M2 to 1.5-final, there are no additional instructions). To upgrade from Daisy 1.5-M2, see the instructions (basically, nothing).

2.12.3.1 Before starting

Shutdown Daisy (the Repository Server, the Daisy Wiki, and the OpenJMS server)

Make backups! More specifically:

daisy-backup-tool -b -d $DAISYDATA_DIR -l $DAISY_BACKUP_DIR -o $DAISY_HOME/openjms -a additional-entries.xml

The '-a additional-entries.xml' is not required to make a backup, but if specified you can backup daisywiki specific files.  If you want to backup these files you must first create the addition-entries.xml file, an example can be found in the backup documentation.

If you can't get the backup tool to work (it was buggy in Daisy 1.4 on Windows), you can also do a manual backup:

2.12.3.2 Download and extract installation files

  1. Rename your existing <DAISY_HOME> directory. During these instructions, we will refer to this location as <OLD_DAISY_HOME>.
  2. Download the Daisy 1.5 installation files from here.
  3. Extract the installation archive. Make sure that  your <DAISY_HOME> variable points to that newly created directory, if necessary, move or rename that directory (or adapt your <DAISY_HOME> environment variable).

2.12.3.3 Switching from OpenJMS to ActiveMQ

2.12.3.3.1 Make sure no messages are left in OpenJMS

Before moving to ActiveMQ, it is a good idea to check if there are no messages waiting to be processed in OpenJMS's queues. Usually JMS messages in Daisy are processed quickly, so unless you have had high activity right before shutting down the Daisy repository, there will normally be no more messages waiting to be processed.

To be sure, you can do a quick check like this:

  1. start OpenJMS
  2. go to OPENJMS_HOME/bin, and execute the "admin" script. This will launch a GUI window. In its menu, choose Actions, Connections, Online. It will ask for a username and password. The username is normally "admin", the password can be found in the file OPENJMS_HOME/config/openjms.xml (look for <User name="admin" password="something"/>). Once connected, you see the topics with their durable subscribers, and the queues. Next to them a number is displayed, they should all be zero.

If there would still be messages, just start the repostory server again, and wait till they have all been processed.

2.12.3.3.2 Creating a database for ActiveMQ

Start the MySQL client:

mysql -uroot

and execute:

CREATE DATABASE activemq;
GRANT ALL ON activemq.* TO activemq@'%' IDENTIFIED BY 'activemq';
GRANT ALL ON activemq.* TO activemq@localhost IDENTIFIED BY 'activemq';

ActiveMQ will automatically create its database tables the first time the repository server is launched.

2.12.3.3.3 ActiveMQ configuration

Create the ActiveMQ configuration :

cp <DAISY_HOME>/repository-server/conf/activemq-conf.xml.template <DAISY_DATA>/conf/activemq-conf.xml
cp <DAISY_HOME>/repository-server/conf/login.config <DAISY_DATA>/conf/
cp <DAISY_HOME>/repository-server/conf/users.properties <DAISY_DATA>/conf/
cp <DAISY_HOME>/repository-server/conf/groups.properties <DAISY_DATA>/conf/

Update myconfig.xml (<DAISY_DATA>/conf/myconfig.xml):

Look for the following element:

<target path="/daisy/jmsclient/jmsclient">

and replace the <configuration> element inside it with:

<configuration>
  <jmsConnection>
    <clientId>daisy-repository</clientId>
    <credentials username="admin" password="jmsadmin"/>
    <initialContext>
      <property name="java.naming.provider.url" value="vm://DaisyJMS?brokerConfig=xbean:file:${daisy.datadir}/conf/activemq-conf.xml"/>
      <property name="java.naming.factory.initial" value="org.apache.activemq.jndi.ActiveMQInitialContextFactory"/>
      <property name="queue.fullTextIndexerJobs" value="fullTextIndexerJobs"/>
      <property name="topic.daisy" value="daisy"/>
    </initialContext>
    <connectionFactoryName>ConnectionFactory</connectionFactoryName>
  </jmsConnection>
</configuration>

If you are using MySQL 5 (thus not 4.1), then you need to edit <DAISY_DATA>/conf/activemq-conf.xml. Look for the following line (here split on two lines for readability):

<property name="url" value="jdbc:mysql://localhost/activemq?relaxAutoCommit=true&amp;
                         useServerPrepStmts=false&amp;sessionVariables=storage_engine=InnoDB"/>

In this, you should remove the "useServerPrepStmts=false&amp;", so that it becomes:

<property name="url" value="jdbc:mysql://localhost/activemq
          ?relaxAutoCommit=true&amp;sessionVariables=storage_engine=InnoDB"/>

2.12.3.4 Configuring the Driver Registrar component

This new component registers JDBC drivers that are used by datasources in the repository-server.

Background info: previously, the JDBC drivers were configured as part of the datasource, but since JDBC connections are now needed by multiple datasources in the same VM (both the repository server and ActiveMQ), their registration is now moved into a specific component.

Edit <DAISY_DATA>/conf/myconfig.xml, and add a new target :

<target path="/daisy/driverregistrar/driverregistrar">
  <configuration>
    <drivers>
      <driver>
        <classpath>${daisy.home}/lib/mysql/jars/mysql-connector-java-3.1.12-bin.jar</classpath>
      	<classname>com.mysql.jdbc.Driver</classname>
      </driver>
    </drivers>
  </configuration>
</target>

Notice the upgrade from mysql-connector 3.1.7 to mysql-connector 3.1.12

You may now also remove the references to JDBC drives from the datasource component.  In the same file look for "/daisy/datasource/datasource".  In the configuration of this component look for the "driverClasspath" and "driverClassName" elements.  You may simply remove both these elements like this :

<target path="/daisy/datasource/datasource">
    <configuration>
      <username>daisy</username>
      <password>daisy</password>
      <url>jdbc:mysql://localhost/daisyrepository?useServerPrepStmts=false</url>
      <driverClasspath>REMOVE THIS LINE</driverClasspath>
      <driverClassName>REMOVE THIS LINE</driverClassName>
      <maxActive>20</maxActive>
      <maxIdle>8</maxIdle>
      <minIdle>0</minIdle>
      <maxWait>5000</maxWait>
    </configuration>
  </target>

2.12.3.5 Upgrade blobstore to hierarchical format

The blobstore directory structure has changed from being flat to being hierarchical, therefore a conversion must be made.  There is a tool that will convert your existing data to the correct format.  It can be executed by running the following command :

<DAISY_HOME>/bin/daisy-blobstore-convertor <path-to-blobstore>

The <path-to-blobstore> is normally <daisy data dir>/blobstore

The tool will print lines like:

Upgrading 1014764c672c25335146498e33393b30642a4207

for each processed file. If the tool outputs nothing, then either something is wrong (you give the incorrect path to the blobstore), or your blobstore is empty (very unlikely, impossible if you've installed the wiki).

2.12.3.6 Run database upgrade script

cd <DAISY_HOME>/misc
mysql -Ddaisyrepository -udaisy -p<password>
[then on the mysql prompt]
\. daisy-1_4-to-1_5_M1.sql

2.12.3.7 Create pubreqs directory and update myconfig.xml

In the daisy data directory, create a new empty subdirectory called pubreqs (all lowercase). This directory becomes thus a sibling of the blobstore, indexstore, conf and logs directories.

Then open the following file in a text editor:

<daisy data dir>/conf/myconfig.xml

And add the following as a child of the root <targets> element (the order of this and the other <target> elements does not matter) 

  <target path="/daisy/extensions/publisher/publisher">
    <configuration>
      <repositoryUser login="internal" password="defaultpwd"/>
      <publisherRequestDirectory>/somewhere/pubreqs</publisherRequestDirectory>
    </configuration>
  </target>

Change the value of the password attribute to the same value as the password attribute on the other <repositoryUser> elements in the same file.

Change the content of the <publisherRequestDirectory> to contain the absolute path to the new pubreqs directory you just created.

2.12.3.8 Updating wrapper configuration for the Repository Server

If you are using the service wrapper (or custom startup scripts) to start the repository server, then you will need to upgrade the configuration/scripts to pass some additional parameters. In dsy_repo_wrapper.conf, these have to be added (see also the full script): 

wrapper.java.additional.4=-Djava.awt.headless=true
wrapper.java.additional.5=-Dorg.apache.commons.logging.Log=org.outerj.daisy.logging.DaisyLog
wrapper.java.additional.6=-Djava.security.auth.login.config=%DAISY_DATADIR%/conf/login.config
wrapper.java.additional.7=-Ddaisy.datadir=%DAISY_DATADIR%
wrapper.java.additional.8=-Ddaisy.home=%DAISY_HOME%

2.12.3.9 Optional: remove absolute paths in myconfig.xml

Daisy now allows to use properties in the myconfig.xml to avoid absolute paths. If you want, you can update your myconfig.xml and change absolute references to the daisy data directory and daisy home directory by ${daisy.datadir} and ${daisy.home} respectively. Doing this will allow to move your data directory around without changing the myconfig.xml.

2.12.4 Start repository server

At this point, start the repository server, as this is needed for the remainder of the upgrade.

It is not needed anymore to start OpenJMS.

2.12.5 Upgrade Daisy Wiki schema

Run the daisy-wiki-init script to update the Daisy Wiki schema types:

cd <DAISY_HOME>/install
daisy-wiki-init

2.12.6 Create wikidata directory

From now on, configurations parameters, skins, doctype XSLs, the book publication types and book store, the wiki log files, in short all 'wiki-instance-specific' stuff is stored in a separate "wikidata directory" (not to be confused with the daisydata directory, which performs a similar role but for the repository server).

We will now create such a directory and copy your existing configuration to it.

To create the wikidata directory, go to <DAISY_HOME>/install and execute:

daisy-wikidata-init

The program will ask you for a location for this directory. Just choose some location for this, for example put it next to (but not inside) the repository daisydata directory.

The program will also ask for the location of the repository daisydata directory, as it will take some configuration values from there (internal & jms user password) and put them in the wiki config.

It will also try to create the registrar user, but since you're doing an upgrade that user will already exist so it will simply skip that. However, you will need to enter the password for the registrar user. This can be found in <OLD_DAISY_HOME>/daisywiki/webapp/WEB-INF/cocoon.xconf, and look there for something like:

<registrarUser login="registrar" password="b6f84c35249a08d11ad6fa8279e2f9"/>

Just copy-and-paste that password to the input prompt.

Once the daisy-wikidata-init script has ended, we can copy over the remaining things:

If you made any configuration changes to the repository manager, JMS client or registrar component which were previously in the cocoon.xconf, then you can now find these in <wikidata dir>/daisy.xconf.

2.12.6.1 Updating wrapper configuration for the Daisy Wiki

If you are using the service wrapper script to launch the Daisy Wiki, you need to adjust the dsy_wiki_wrapper.conf to include the following parameter (see also the full script):

wrapper.java.additional.7=-Ddaisywiki.data=%DAISYWIKI_DATADIR%

Daisy now ships with Jetty 5, which means that the classpath definition in the dsy_wiki_wrapper.conf needs to be updated. Just replace the existing wrapper.java.classpath entries with the new ones found in the updated dsy_wiki_wrapper.conf file.

The DAISYWIKI_DATADIR environment variable has to be declared in the shell script (dsy_wiki.sh or equivalent):

#! /bin/sh

DAISY_HOME=/home/daisy/daisy ; export DAISY_HOME
JETTY_HOME=$DAISY_HOME/daisywiki/jetty ; export JETTY_HOME
DAISYWIKI_DATADIR=/path/to/your/daisywikidata ; export DAISYWIKI_DATADIR

[.....]

2.12.7 Start the Daisy Wiki

Now you can also start the Daisy Wiki.

2.12.8 After-upgrade notes

2.12.8.1 [optional] Image thumbnail generation

Daisy 1.5 can generate image thumbnails and extract their width and height to store them in fields of the Image document type. If you want to do this for all your current images, you can do so by running a document task. For this, first make sure you're logged in as Administrator, then go to the Document Tasks screen (the link is in the user menu) and create a new document task. Select the documents using the query:

select name where documentType = 'Image'

You might first want to try this with one document to see if everything is OK.

Then go to the next screen, and use the following script:

var document = repository.getDocument(variantKey, true);
document.setField("ImageWidth", new java.lang.Long(0));
document.save();

This script "touches" the document by setting a field to a dummy value. Otherwise, the document wouldn't be changed and no save will be performed.

2.12.8.2 OpenJMS cleanup

Since OpenJMS is not used anymore, some cleanup you might want to to:

2.12.8.3 ActiveMQ authentication

With the procedure we followed above, ActiveMQ still uses default passwords. If the ActiveMQ port is publicly accessible, it is of course very much recommended to change these. Here is how. The password needs to be changed in the following locations:

Original setting:

In  this file:
<datadir>/conf/users.properties

in this line:
admin=jmsadmin

For the connection made by the Repository Server:

In this file:
<datadir>/conf/myconfig.xml

in this line:
<credentials username="admin" password="jmsadmin"/>

For the connection made by the Daisy Wiki:

In this file:
<wikidata dir>/daisy.xconf

in this line:
<credentials username="admin" password="jmsadmin"/>

After making these changes, stop both the Daisy Wiki and the Repository Server, and then restart them (first the repository, then the wiki).

2.12.8.4 Note about static resources and caching

Since static resources are cached for about 5 hours by your browser, you might need to clear the browsers' cache.

In Firefox, this is done using Edit -> Preferences -> Privacy -> press the "Clear" button next to Cache

Internet Explorer users, see here.

If these instructions or unclear to you, or if you find an error in them, please share them with us on the Daisy mailing list or by leaving a comment.

2.13 1.5 to 1.5.1 upgrade

2.13.1 Changes

If you are currently using Daisy 1.5.0, there is little reason to upgrade to this release. No changes have be done except for those listed below.

2.13.2 Upgrading

To upgrade from releases earlier then 1.5.0, follow the same instructions as for upgrading to 1.5.0 (e.g. upgrade from 1.4 to 1.5).

Nothing special needs to be done to upgrade from Daisy 1.5.0 to this release.

Just stop the repository and wiki servers, extract the new release, adjust DAISY_HOME or move the old Daisy install out of the way, and restart the servers.

3 Source Code

Sources can be obtained through SVN. Instructions for setting up a development environment with Daisy (which is slightly different from using the packaged version) are included in the README.txt's in the source tree. For anonymous, read-only access to Daisy SVN, use the following command:

svn co http://svn.cocoondev.org/repos/daisy/trunk/daisy

This will give the latest development code (the "trunk"). To get the source code of a specific release, use a command like this:

svn co http://svn.cocoondev.org/repos/daisy/tags/RELEASE_1_3_1 daisy

See also the existing tags.

No authentication is required for anonymous access. If you're behind a (transparent) proxy, you might want to verify whether your proxy supports the extended HTTP WebDAV methods.

4 Repository server

4.1 General

4.1.1 Documents

4.1.1.1 Introduction

The purpose of the Daisy Repository Server is managing documents. This document will describe the structure (or features) of such documents.

The diagram below gives an overview of the document structure, this will be further explained in the remainder of this document.

4.1.1.2 Documents & Document Variants

A document in itself has very little properties, the real meat is in the document variants. A document never exists without at least one document variant. On the other hand, making explicit use of variants is optional, in which case you could consider a document and a document variant to be the same (thus, each document then has exactly one variant).

The details of working with variants are described in another section. For now, it suffices to know that in a practical working environment like the Daisy Wiki, the branch and language which identify the particular variant of the document are usually a given (Daisy Wiki: configured per site), and you'll only work with document IDs, so it is as if the existence of variants is transparent.

Many times when we speak about a document in Daisy, we implicitly mean "a certain variant of a document" (a "document variant").
Refer to the diagram above to see if a certain aspect applies to a document, a document variant, or a version of a document variant.

A document is always retrieved from the repository in combination with a document variant, a document in itself without a variant cannot be retrieved. This is, among other things, because the access rights to a document are based on information that is part of the document variant (it can thus be that a user has access to one document variant but not to another). Another way to look at this is that there are only document variants, and that certain properties of them are shared across the variants.

4.1.1.3 Document Types, The Repository Schema

The main "data" of a document is contained in its so-called parts and fields. Parts can contain arbitrary binary data, and fields contain 'simple' information of a certain type (string, date, decimal, ...). Which parts and fields a document can have is determined by the document's document type. A document type is actually a combination of zero or more part types and zero or more field types, which further describe these aspects. Part and field types are defined as independent entities, meaning that the same part and field types can be reused across different document types. The diagram below shows the structure and relation of all these entities.

4.1.1.3.1 Common aspects of document, part and field types

Let us first look at the things document, part and field types have in common. Their primary, unchangeable identifier is a numeric ID, though they also have a unique name (which can be changed after creation), which you will likely prefer to use.

Next to the name, they can be optionally assigned a localized label and a description. Localized means that a different label and description can be given for different locales. A locale can be a language, language-country, or language-country-variant specification. For example, a label entered for "fr-BE " would mean it is in French, and specifically for Belgium. The labels and descriptions are retrieved using a fallback system. For example, if the user's locale is "fr-BE", the system will first check if a label is available for "fr-BE", if not found it will check for "fr", and finally for the empty locale "". Thus if you want to provide labels and descriptions but are not interested in localisation, you can simply enter them for the empty locale.

Document, part and field types cannot be deleted as long as they are still in use in the repository. Once a document has been created that uses one of these types, the type can thus not be deleted anymore (unless the documents using them are deleted). However, it is possible to mark a type as deprecated to indicate it should not be used anymore. This deprecation flag is purely informational, the system simply stores it.

4.1.1.3.2 Document types

A document type combines a number of part types and field types, and indicates for each of these if it is required or not.

4.1.1.3.3 Part types

Before going into the details of part types, it might make sense to justify their existence. In many document repository systems, each document has simply one 'content chunk'. For example, a resource addressable over webdav is one atomic piece of data. Daisy allows a document to consist of multiple parts. This makes these parts separately addressable and retrievable. For example, suppose we have a document type consisting of a part "Abstract" and a part "Main Content". It is then simple to retrieve the abstracts of all documents conforming to this document type. As another example, for an "Image" document type we could have parts "ImageData" containing a rendered form of the image, "ImageSource" containing the original source (e.g. a Photoshop or CorelDraw file), and "Thumbnail" containing a small rendition of the image.

A part instance consists of some binary data (or if you wish, data which is treated as binary, it could be plain text of course), and the mime-type of the data. A part type allows to restrict which types of data (thus which mime-types) are stored in the part, but this is not required. This restriction is done by specifying a list of allowed mime types.

4.1.1.3.3.1 The Daisy HTML flag

A part type has a flag indicating whether the part contains "Daisy HTML". Daisy HTML is basically HTML formatted as well-formed XML (with element and attribute names lowercased). It is not the same as XHTML, because the elements are not in the XHTML namespace. If the "Daisy HTML" flag is set to true, the mime-type should be limited to text/xml. For the repository server, the Daisy-HTML flag on the part type has little meaning. Currently it serves only to enable the creation of document summaries (which might even be replaced with a more flexible mechanism in the future). The Daisy Wiki front end application will show a wysiwyg editor for Daisy HTML parts, and display the content of such parts inline.

4.1.1.3.3.2 Link extraction

For each part type a link extractor can be defined to extract links from the content contained in the part. The most common link extractor is the "daisy-html" one, which will extract links from the href attribute of the <a> element, the src attribute of the <img> element, and the character content of <p class="include">. The format of the links is:

daisy:<document id>
or
daisy:<document id>@<branch id or name>:<language id or name>:<version id>#fragment_id

Links that don't conform to this form will be ignored. The <version id> can take the special value "LAST" (case insensitive). A link without a version specification denotes a link to the live version of the document. The branch, language and version and fragment ID parts are all optional. For example, daisy:15@:nl is a link to the Dutch version of document 15.

4.1.1.3.4 Field types
4.1.1.3.4.1 Value Type

The most important thing a field type tells about a field is its Value Type. A Value Type identifies the kind of data that can be stored in a field, the available value types are listed in the table below, together with their matching Java class.

Value Type Name

Corresponding Java class

string

java.lang.String

date

java.util.Date

datetime

java.util.Date

long

java.lang.Long

double

java.lang.Double

decimal

java.math.BigDecimal

boolean

java.lang.Boolean

link

org.outerj.daisy.repository.VariantKey

The link type is somewhat special: it defines a link to another document variant. Its value is thus a triple (document ID, branch ID, language ID). The branch ID and language ID are optional (value -1 in the VariantKey object) to denote they should default to the same as the containing document.

4.1.1.3.4.2 Multi-Value

A field type can specify that it concerns a multi-value field, thus that fields of that type can have multiple values. All of the values of the field should be of the same value type.

A multi-value field can have more than once the same value, and the order of values of a multi-value field is maintained. Thus the values of a multi-value field form an ordered list.

4.1.1.3.4.3 Selection Lists

It is possible to define a selection list for a field type. This is a list of possible values that an end user can choose from when completing the field. There are multiple available selection lists types:

4.1.1.3.4.4 ACL allowed flag

In the access control system, it is possible to define access rules for documents by using an expression to select the documents to which the access rules apply. In these expressions, it is also possible to check the value of fields, but only of fields whose field types' ACL allowed flag is set to true. The ACL allowed flag also enables the front-end to indicate that changing the value of that particular field can influence the access control checks.

4.1.1.3.4.5 Size hint

A field can have a size hint, this is simply an integer number. This information is used by the front end to display an input field of an appropriate width. The repository server doesn't associate any further meaning to it, it doesn't cause any validation to happen, nor does it specify the unit of the width (most likely to be "number of characters").

4.1.1.3.5 Document and document type association, how changes to document types are handled

Upon creation of a document, a document type must be supplied. When saving a document, the repository will check that the document conforms to its document type. Thus it will check that all required fields and parts are present, and that there are no parts and fields in the document that are not allowed by the document type.

The document type of a document can be changed at any time. This is useful if you start out with a generic document type but later want to switch to a more specialized document type.

The definition of a document type can be changed at any time. Part and field types can be added or removed from it, or can be made required. A logical question that pops up is what happens to existing documents in the repository that use the changed document type. The answer is basically "nothing". If for example a required field is added to a document type, then the next time a document of that type is edited, it will fail to save unless a value for the field is specified. The newly saved version of the document will then conform to the new state of the document type. Older versions of the document will remain unchanged however. When saving a document, it is also possible to supply an option that tells not to do the document type conformance check.

So basically the document type system doesn't give any guarantees about the structure of the documents in the repository, but rather hints at how the documents should be structured and interpreted.

4.1.1.4 Documents

A document consists of versioned and non-versioned data. Versioned data means that each time the document is saved (and some of the versioned aspects of the document changed), a new version will be stored, so that the older state of the data can still be viewed afterwards. In other words, it provides a history of who made what changes at what time.

When a document is saved for the first time, it is assigned a unique, numeric ID. The ID is just a sequence counter, so the first created document gets ID 1, then 2, and so on. The ID of a document never changes. The user who creates the document is the owner of the document. The date and time of document creation is also stored.

When creating a document, its document type must be specified. The document type can afterwards be changed.

Daisy has no directories like a filesystem. Everything is just in one big bag. When saving a document, you only have to choose a name for it (which acts in fact as the title of the document), and this name is not even required to be unique (see below). Documents are retrieved by searching, or browsing through navigation trees.

4.1.1.4.1 Versioned Content

The versioned content of a document consists of the following:

So if any changes are made to any of these, and the document is stored, a new version is created.

4.1.1.4.1.1 Version ID

Each version has an ID, which is simply a numeric sequence number: the first version has number 1, the next number 2, and so on.

4.1.1.4.1.2 Document Name

The name of a document is required (it cannot be empty). The name is not required to be unique. Thus there can be multiple documents with the same name. The ID of the document is its unique identification.

The name is usually also rendered as the title of the document.

4.1.1.4.1.3 Parts

The parts. Each part is associated with a part type and has a mime type and some data. There cannot be two parts of the same part type in one document.

Each part can optionally have a file name, this file name can be used as default file name when the content of the part is saved (downloaded) in a file.

4.1.1.4.1.4 Fields

The fields. Each field is associated with a field type and specifies the field value. There cannot be two fields of the same field type in one document.

4.1.1.4.1.5 Links

A document can contain two kinds of links: links can occur as content of a part (for example, an <a> element in HTML), and a document can have a number of so-called out-of-line links. These are links stored separately from the content. Each link consists of a title and a target (some URL). These links are usually rendered at the bottom of a page in as a bulleted list.

Out-of-line links are useful in case you want to link to related documents and either don't want or can't (e.g. in case of non-HTML content) link to them from the content of a part.

4.1.1.4.1.6 Version state & the live version

Each version can have a state indicating whether it is a draft version (i.e. you started editing the document but are not finished yet, in other words the changes should not yet be published), or a publishable version. The most recent version having the state 'publish' becomes the live version. The live version is the version that is shown by default to the user. It is also the version whose data is indexed in the full-text index, and whose properties are used by default when querying.

4.1.1.4.2 Non-versioned properties
4.1.1.4.2.1 Collections and collection membership

Collections are sets of documents. A document can belong to one or more collections, thus collections can overlap. A collection is simply a way to combine some documents in order to do something with them or treat them in some special way.

Collections themselves can be created or deleted only by Administrators (in the Daisy Wiki, this is done in the administration interface). Deleting a collection does not delete the documents in it. You can limit who can put documents in a collection by ACL rules.

4.1.1.4.2.2 Custom fields

Custom fields are arbitrary name-value pairs assigned to a document. The name and value are both strings. In contrast with the earlier-mentioned fields that are part of the document type, these fields are non-versioned. This makes it possible to stick tags to documents without causing a new version to be created, and without formally defining a field type.

4.1.1.4.2.3 Private

A document marked as private can only be read (and written) by its owner.

While the global access control system of Daisy makes it easy to centrally handle access control for sets of documents, sometimes it could be useful to simply say "I want nobody else to see this (for now)". This can be done by enabling the private flag. The document will then not be accessible for others, and also won't turn up in search results done by others. The private flag can be set on or off at any time, by the owner or by an Administrator.

There is however one big exception: Administrators can always access all documents, and thus will be able to read your "private" documents. The content is not encrypted.

4.1.1.4.2.4 Retired

If a document variant is no longer needed, because its content is outdated, replaced by others, or whatever, you can mark the document variant as retired. This makes the document variant virtually deleted. It won't show up in search results anymore.

The retired flag can be set on or off at any time, retiring is not a one-time operation.

4.1.1.4.2.5 Lock

A lock can be taken on a document variant to make sure nobody else edits the document variant while you're working on it.

Daisy automatically performs so-called optimistic locking, this means that if person A starts editing the document, and then person B starts editing the document, and then person A saves the document, and then person B tries to save the document, this last operation will fail because the document has changed since the time person B loaded it. This mechanism is always enabled, it is not needed to take an explicit lock.

A lock can then be taken to make others aware that you are editing the document. A lock can be of two types: an exclusive lock or a warn lock. An exclusive lock is pretty much as its name implies: it is a lock exclusively for the user who requested it, and avoids that any one else will be able to save the document until you release the lock. A warn lock then isn't really a lock, it is just an informational mechanism to let others know that someone else also started to edit the document, but it doesn't enforce anything. Anyone else can still at any time save the document or replace the lock with their own.

A lock can optionally have a certain duration, if the duration is expired, the lock is automatically removed.

For example, the Daisy Wiki application by default uses exclusive locks with a duration of 15 minutes, and automatically extends them when the user keeps editing.

A lock can be removed either by the person who created it, or by an Administrator.

4.1.1.4.2.6 Owner

The owner of a document is a person who is always able to access (read/write) the document, regardless of what the ACL specifies. The owner is initially the creator of the document, but can be changed afterwards.

4.1.1.4.2.7 Last Modified and Last Modifier

Each time a document is saved, the user id of the person who saved it is stored as the last modifier, and the date and time of the save operation as the "last modified" time. Each document variant also has their own Last Modified and Last Modifier information. For document variants, this will often fall together with the Created/Creator fields of the last version, but not necessarily so: if only non-versioned properties are changed, no new version will be created.

4.1.2 Variants

4.1.2.1 Introduction

The variants feature of Daisy allows to have multiple alternatives of a document stored in one logical document, thus identified by one unique ID.

Daisy allows to have variants among two axes:

For example, if there would not be a variants feature, and you had the same content in different languages, for each of these languages you would need to create a different document, thus with a different ID.

Language variants are quite obvious, but you may wonder what branches are. The purpose of branches is to have multiple parallel editable versions of the same content. As an example, take the Daisy documentation. Between major Daisy releases there might be quite some changes to the documentation. However, while creating the documentation of e.g. Daisy 1.3, we still want the ability to update the documentation of Daisy 1.2. Sure, this could be solved by duplicating all documentation documents for each new release, but then the identity of these documents would be lost since they get new IDs assigned, and the relationship between the documents in different releases would be lost.

4.1.2.2 Defining variants

By default, Daisy predefines one branch and one language variant: the branch main and the language default.

You can yourself define other ones, in the Daisy Wiki you can do this via the administration screens.

The definition of a branch or language consists of a numeric ID (assigned by the repository server), a name and optionally a description. Internally, the ID is used, but towards the user mostly the name is shown.

The built-in main branch and default language each have as ID 1.

Once a branch and/or language is defined, you can create new document variants using them.

Defining the branches and languages is something that can only be done by users who have the Administrator role, but adding variants to documents (which is almost the same as creating documents) can of course be done by any user, as far as the ACL allows the user to do so.

Deleting a branch or language definition is only possible when there are no more document variants for that branch or language. You can easily delete all document variants for a certain branch or language using the Document Task Manager, similarly to what is described further on for creating a variant across a set of documents.

4.1.2.3 Creating a variant on a document

When adding a new variant to a document, this can be done in two ways:

  1. from scratch
  2. based on the content of (a certain version of) an existing variant

When you opt for the second option (which is mostly done when creating branch-variants) then the (branch,language,version)-triple from which the content is taken will be stored as part of the new variant, so that later on you can see from where this variant "branched" (in the Daisy Wiki, this information is shown on the version list page).

In the Daisy Wiki, there is an "Add Variant" action that allows to add a new variant to a document.

4.1.2.4 Searching for non-existing variants

When translating a site, it can be useful to search which documents are not yet translated in a certain language. Similarly, it can be useful to see which documents exist on one branch but not on another. For this purpose, the query language provides a function called DoesNotHaveVariant(branch, language).

For example, to search on the Daisy site for all documents that have been added in the documentation of version 1.3 compared to 1.2, you can use the following query:

select id, name
  where
   InCollection('daisydocs')
   and branch = 'daisydocs-1_3' and language = 'en'
   and DoesNotHaveVariant('daisydocs-1_2', 'en')

4.1.2.5 Queries embedded in documents

When using queries embedded in documents together with variants, usually you will want to limit the query results to variants with the same branch and language as the one containing the query. You could specify these explicitly, as in:

select id, name where <conditions> and branch='my_branch' and language='my_lang'

However, this means that you will need to adjust these queries when adding new variants to the document. Especially if you are adding a certain branch to a set of documents, this is not something you want to do. Therefore, it should be possible to refer to the branch and language of the containing document. This can be done as follows:

select id, name where <conditions> and branchId = ContextDoc(branchId) and languageId = ContextDoc(languageId)

4.1.2.6 Creating a variant across a set of documents

When using branches, you will often want to add a variant for that branch to a set of documents (in other words: create a branch across a set of documents). To avoid the need to do this one-by-one for each document, Daisy has a "Document Task Manager" which allows the execution of a certain task on a set of documents. And that task could for example be "adding a new variant".

The Document Task Manager is covered in a separate section, here we will just focus on how to use it to create a new variant.

Before using the Document Task Manager, be sure you have defined the new branch (or language) using the administration screens.

In the Daisy Wiki, the Document Task Manager is accessed via the drop-down User-menu (in the main navigation bar). Select the option to create a new task. You are then first presented with a screen where you need to specify the documents (document variants actually) with which you want to do something. As you can see, it is possible to add documents using queries. For example, for the Daisy site, when we want to create a branch starting from the Daisy 1.2 documentation, we would use a query like:

select id, name where InCollection('daisydocs') and branch = 'daisydocs-1_2' and language = 'en'

Once you selected the documents, press Next to go to the next page where the action to be performed on the documents is specified. For Type of task choose Simple Actions. Then press the Add button to add a new action. Change the type of the action to Create Variant (if necessary), and specify the branch and language you want to create. Finally press start to start the task. You can then follow up on the progress of this operation, and check if it finished successfully for all documents.

4.1.3 Document Comments

This section is about document comments: comments that can be added to Daisy documents. More precisely, they are actually added to document variants, thus each variant of a document has its own comments.

4.1.3.1 Comment features

The current Daisy comments system is rather simple (text-only comments, no editing after creation, no threading) but nonetheless very useful.

4.1.3.1.1 Comment visibility

Each comment has a certain visibility:

4.1.3.1.2 Creation of comments

Everyone who has read access to a document can add comments to it. Editors-only comments can however only be created by users with write access to the document.

4.1.3.1.3 Deletion of comments

Comments can be removed from a document by the users who have write access to the document (this includes users acting in the Administrator role). Private comments can be deleted by its creator, independent of whether that user has write access to the document the comment belongs too.

When a document is deleted, all its associated comments are removed too, including private ones that the deleter of the document may not be aware of.

4.1.3.2 Daisy Wiki specific notes

4.1.3.2.1 Guest user cannot create comments

The guest user, though it is for the repository server an ordinary user like any other, is not allowed to create comments via the Daisy Wiki. This means that to create comments, users should first log in.

4.1.3.2.2 'My Comments' page

Users can get a list of all the private comments they added to documents via a "My Comments" page (accessible via the drop-down menu behind the user name).

4.1.4 Query Language

4.1.4.1 Introduction

The Daisy Query Language can be used to search for documents (more precisely, document variants). In the Daisy Wiki, queries can be used in various places:

The implementation of various Daisy Wiki features is also based on queries, such as the recent changes page or the referrers page. And of course it is possible to execute queries from your own applications, using the HTTP interface or Java API.

The query language is a somewhat SQL-like language that allows to search on various document properties (including the fields), fulltext on the part content, or a combination of those. The sort order of the results can also be defined. The resulting document list is filtered to only include documents to which the user has at least read-live access.

An example query, searching all documents in a collection call "mycollection":

select id, name where InCollection('mycollection') order by name

Internally, non-fulltext queries are translated to SQL and executed on the relational database while fulltext queries are executed by Jakarta Lucene.

Although the query language is somewhat SQL-like, it hides the complexity of the actual SQL-queries that are performed by the repository server on the relational database, which can quickly grow quite complex.

Note: everytime in this document when we talk about "searching documents", this is equivalent to "searching document variants". The result of query is a set of document variants, i.e. each member of the result set is identified by a triple (document ID, branch, language).

4.1.4.2 Query Language

4.1.4.2.1 General structure of a query
select
  ...
where
  ...
order by
  ...
limit x
option
  ...

The select and where parts are required, the rest is optional. Whitespace is of no importance.

4.1.4.2.2 The select part

The select part should list one or more value expressions, separated by commas. A value expression can be an identifier, a literal or a function call. This is described in more detail further on.

4.1.4.2.3 The where part

The where part should contain a predicate expression, thus an expression which tests the value of value expressions using operators, or uses some built-in conditions.

Besides the operators listed in the table below, the operations AND and OR are supported, and parentheses can be used for grouping.

4.1.4.2.3.1 Operators & data types

string

long

double

decimal

date

datetime

boolean

=

X

X

X

X

X

X

X

!=

X

X

X

X

X

X

X

<

X

X

X

X

X

X

>

X

X

X

X

X

X

<=

X

X

X

X

X

X

>=

X

X

X

X

X

X

[NOT] LIKE

X

[NOT] BETWEEN

X

X

X

X

X

X

[NOT] IN

X

X

X

X

X

X

IS [NOT] NULL

X

X

X

X

X

X

X

Wildcards for LIKE are _ and %, escape using \_ and \%.

All keywords such as AND, LIKE, BETWEEN, ... can be written in either uppercase or lowercase (but not mixed case).

If these operators are used on multi-value fields, they return true if at least one of the values of the multi-value field satisfies. See further on for a set of conditions specifically for multi-value fields.

4.1.4.2.4 Value expressions

A value expression is:

A function call usually has the following form:

functionName(arg1, arg2, ...)

However, for the basic mathematical functions (addition, subtraction, multiplication and division) "infix" notation is used instead, using the symbols +, -, * and /. Parentheses can be used to influence the order of the operations.

4.1.4.2.5 Identifiers

The table below lists the available identifiers.

Some notes:

name

searchable

datatype

version dependent

remarks

id

yes

long

no

name

yes

string

yes

branch

yes

symbolic

no

branchId

yes

long

no

language

yes

symbolic

no

languageId

yes

long

no

link

yes

link

no

The current document variant as a link. This is useful for comparison with link type fields. For example $someLinkField = link to find documents which link to themselves in a certain field, or $someLinkField = ContextDoc(link) to find documents which link to the context document.

documentType

yes

symbolic

no

versionId

yes

long

yes

ID of the live version, or if the query option search_last_version is specified, of the last version

creationTime

yes

datetime

no

ownerId

yes

long

no

ownerLogin

yes

symbolic

no

ownerName

no

string

no

summary

no

string

no

always of last published version

retired

yes

boolean

no

private

yes

boolean

no

lastModified

yes

datetime

no

lastModifierId

yes

long

no

lastModifierLogin

yes

symbolic

no

lastModifierName

no

string

no

variantLastModified

yes

datetime

no

variantLastModifierId

yes

long

no

variantLastModifierLogin

yes

symbolic

no

variantLastModifierName

yes

string

no

%partTypeName.mimeType

yes

string

yes

%partTypeName.size

yes

long

yes

%partTypeName.content

no

xml

yes

only works for part types for which the flag 'daisy html' is set to true, and additionally the actual part must have the mime type 'text/xml'

versionCreationTime

yes

datetime

yes

versionCreatorId

yes

long

yes

versionCreatorLogin

yes

symbolic

yes

versionCreatorName

yes

string

yes

versionState

yes

symbolic

yes

'draft' or 'publish'

totalSizeOfParts

yes

long

yes

sum of the size of all parts in document

versionStateLastModified

yes

datetime

yes

lockType

yes

symbolic

no

'pessimistic' or 'warn'

lockTimeAcquired

yes

datetime

no

lockDuration

yes

long

no

(in milliseconds)

lockOwnerId

yes

long

no

lockOwnerLogin

yes

symbolic

no

lockOwnerName

no

string

no

collections

yes

symbolic

no

The collections (the names of the collections) the document belongs too. Behaves the same as a multi-value field with respect to applicable search conditions.

collections.valueCount

yes

symbolic

no

The number of collections a document belongs too.

$fieldTypeName

yes

yes

datatype depends on field type

$fieldTypeName.valueCount

yes

long

yes

Useful for multi-value fields. Searching for a value count of 0 does not work, use the "is null" condition instead.

$fieldTypeName.documentId

yes

long

yes

These special field sub-identifiers are only supported on fields of the type "link". For link field types, the $fieldTypeName identifier checks on the document ID, while these identifiers can be used to check on the branch and language.

$fieldTypeName.branch

yes

symbolic

yes

$fieldTypeName.branchId

yes

long

yes

$fieldTypeName.language

yes

symbolic

yes

$fieldTypeName.languageId

yes

long

yes

#customFieldName

yes

string

no
4.1.4.2.6 Literals
4.1.4.2.6.1 String literals

Strings (text) should be put between single quotes, the single quote is escaped by doubling it, for example:

'''t is mooi weer vandaag'
4.1.4.2.6.2 Numeric literals

These consists of digits (0-9), the decimal separator is a dot (.).

Numeric literals can be put between single quotes like strings, but it is not required to do so.

4.1.4.2.6.3 Date & datetime literals

Date format: 'YYYY-MM-DD'

Datetime format: 'YYYY-MM-DD HH:MM:SS'

4.1.4.2.6.4 Link literals

When searching on fields of type "link", the link should be specified as:

'daisy:docid'               (assumes branch main and language default)
'daisy:docid@branch'        (assumes language default)
'daisy:docid@branch:lang'   (branch can be left blank which defaults to main branch)

Branch and language can be specified either by name or ID.

So a search condition could be for example:

$someLinkField = 'daisy:35'
4.1.4.2.7 Special conditions for multi-value fields
$fieldName has all (value1, value2, value3, ...)

Tests that the multi-value field has all the specified values (and possibly more).

$fieldName has exactly (value1, value2, value3, ...)

Tests that the multi-value field has all the specified values, and none more. The order is not important.

$fieldName has some (value1, value2, value3, ...)
or
$fieldName has any (value1, value2, value3, ...)

has some and has any are synonyms. They test that the multi-value field has at least one of the specified values.

$fieldName has none (value1, value2, value3, ...)

Tests that the multi-value field has none of the specified values.

In addition to these conditions, you can use is null and is not null to check if a document has a certain (multi-value) field. The special sub-identifier $fieldName.valueCount can be used to check the number of values a multi-value field has.

4.1.4.2.8 Other special conditions
4.1.4.2.8.1 InCollection
InCollection('collectionname' [, collectionname, collectionname])

Searches documents contained in at least one of the specified collections. To search documents that occur in multiple collections (thus in the intersection of those collections), use the function InCollection multiple times with AND in between: InCollection('collection1') and InCollection('collection2'). This also works for OR but in that case it is more efficient to give the collections as arguments to one InCollection call.

Instead of the InCollection condition, you can use the collections identifier in combination with the multi-value field search conditions such as has some, has all or has none for more powerful search possibilities. The InCollection condition predates the existence of multi-value fields, but remains supported.

4.1.4.2.8.2 LinksTo, LinksFrom, LinksToVariant, LinksFromVariant
LinksTo(documentId, inLastVersion, inLiveVersion [, linktypes])
LinksFrom(documentId, inLastVersion, inLiveVersion [, linktypes])
LinksToVariant(documentId, branch, language, inLastVersion, inLiveVersion [, linktypes])
LinksFromVariant(documentId, branch, language, inLastVersion, inLiveVersion [, linktypes])

Searches documents which link to or from the specified document (or document variant). The other two parameters, inLastVersion and inLiveVersion, are interpreted as booleans: 0 is false, any other (numeric) value is true.

If inLastVersion is true, only documents whose last version link to the specified document are included.

If inLiveVersion is true, only documents whose live version link to the specified document are included.

If both parameters are true or both are false, all documents are returned for which either the last or live version link to the specified document.

The optional parameter linktypes is a string containing a whitespace separated list of the types of links to include, which is one or more of: inline, out_of_line, image, include or other.

4.1.4.2.8.3 IsLinked, IsNotLinked
IsLinked()
IsNotLinked()

IsLinked() evaluates to true for any document which is linked by other documents, IsNotLinked() evaluates to true for any document that is not linked from any other document (thus not reachable by following links in documents, the navigation tree, or linked by the content of other parts on which link extraction is performed).

4.1.4.2.8.4 HasPart
HasPart('partTypeName')

Searches documents which have a part of the specified part type. This search is version-dependent.

4.1.4.2.8.5 HasPartWithMimeType
HasPartWithMimeType('some mimetype')

Searches documents having a part with the given mime type. This search is version-dependent. This uses a 'like' condition, thus the % wildcard can be used in the parameter. For example, to search all images: HasPartWithMimeType('image/%')

4.1.4.2.8.6 DoesNotHaveVariant
DoestNotHaveVariant(branch, language)

Searches documents that do not have the specified variant. See also the page on variants for more information.

4.1.4.2.9 Functions

The following functions can be used in value expressions.

4.1.4.2.9.1 String functions
4.1.4.2.9.1.1 Concat

Syntax:

Concat(string1, string2, ...) : string

Concatenates multiple strings.

4.1.4.2.9.1.2 Length

Syntax:

Length(string) : long

Returns the length of its string argument.

4.1.4.2.9.1.3 Left

Syntax:

Left(string, length) : string

Returns 'length' leftmost characters from the string. If 'length' is larger than the string, the whole string is returned. If length is 0, an empty string is returned.

4.1.4.2.9.1.4 Right

Syntax:

Right(string, length) : string

Returns 'length' rightmost characters from the string. If 'length' is larger than the string, the whole string is returned. If length is 0, an empty string is returned.

4.1.4.2.9.1.5 Substring

Syntax:

Substring(string, position, length) : string

Returns a string formed by taking 'length' characters from the string at the specified position. The 'length' argument is optional, if not specified, it will go till the end of the input string. The 'position' argument starts at 1 for the first character.

4.1.4.2.9.1.6 UpperCase

Syntax:

UpperCase(string) : string
4.1.4.2.9.1.7 LowerCase

Syntax:

LowerCase(string) : string
4.1.4.2.9.2 Date and datetime functions
4.1.4.2.9.2.1 CurrentDate

Syntax:

CurrentDate(spec?) : date

Returns the current date.

The optional spec argument allows to specify an offset to the current date. It is a string with the following syntax:

+/- <num> (days|weeks|months|years)

For example:

CurrentDate('- 7 days')
4.1.4.2.9.2.2 CurrentDateTime

Syntax:

CurrentDateTime(spec?) : date

Returns the current datetime.

The optional spec argument allows to specify an offset to the current datetime. It is a string with the following syntax:

+/- <num> (seconds|minutes|hours|days|weeks|months|years)

For example:

CurrentDateTime('- 3 hours')
4.1.4.2.9.2.3 Year, Month, Week, DayOfWeek, DayOfMonth, DayOfYear

These functions all take a date or datetime as argument, and return a long value.

DayOfWeek returns a value in the range 1-7, where 1 is sunday.

For the Week function, the first week of the year is the first week containing a sunday.

4.1.4.2.9.2.4 RelativeDate, RelativeDateTime

These functions take one string argument consisting of 3 words, each one taken from the following groups:

start    this     week
end      last     month
         next     year

So for example:

RelativeDate('start this month')

returns a date set to the first day of the current month.

4.1.4.2.9.3 Numeric functions
4.1.4.2.9.3.1 +, -, * and /

The basic mathematical operations.

4.1.4.2.9.3.2 Random

Returns a pseudo-random double value greater than or equal to 0 and less than or equal to 1.

4.1.4.2.9.3.3 Mod

Syntax:

Mod(number1, number2)
4.1.4.2.9.3.4 Abs, Floor, Ceiling

These functions all take one number as argument.

4.1.4.2.9.3.5 Round

Syntax:

Round(number, scale)

Rounds the given number to have at most scale digits to the right of the decimal point.

4.1.4.2.9.4 Special
4.1.4.2.9.4.1 ContextDoc

Syntax:

ContextDoc(expression)

In some cases a context document is available when performing a query. For example, when a query is embedded inside a document, that document serves as the context document. It is possible to evaluate expressions on this context document by use of this ContextDoc function.

Examples:

ContextDoc(id) -- the id of the context document
ContextDoc($someField) -- the value of a field of the context document
ContextDoc(Concat(name, ' ', $someField))
4.1.4.2.9.4.2 UserId

Returns the ID of the current user (= the user executing the query).

4.1.4.2.10 Full text queries

For full text queries, the where part takes a special form. There are two possibilities: either only a full text search is performed, or the fulltext query is further restricted using 'normal' conditions. The two possible forms are:

... where FullText('word')
or
... where FullText('word') AND <other conditions>
for example:
... where FullText('word') AND $myfield = 'abc' AND InCollection('mycollection')

Note that the combining operator between the FullText condition and other conditions is always AND, thus the result of the full text query is further refined. The further conditions can of course be of any complexity, and can thus again contain OR.

The FullText clause needs to be the first after the word "where", it cannot appear at arbitrary positions in the where-clause.

If no order by clause is included when doing a full text query, the results are ordered according to the score assigned by the fulltext search engine.

The parameter of the FullText(...) function is a query which is passed on to the full text engine, in our case Lucene. See here.

The FullText() function can have 3 additional parameters which indicate if the search should be performed on the document name, document content or field content. By default, all three are searched. These parameters should be numeric: 0 indicates false, and any other value true.

For example:

FullText('word', 1, 0, 0)

Searches for 'word', but only in the document name.

Additionally, you can specify a branch and language as parameters to the FullText function, to specify that only documents of that branch/language should be searched. Thus the full syntax of the FullText function is:

FullText(lucene query, searchInName, searchInContent, searchInFields, branch, language)

Specifying the branch and language as part of the FullText function is more more efficient then using:

FullText(lucene query) and branch = 'my_branch' and language = 'my_language'
4.1.4.2.11 The order by part

The order by part is optional.

The order by part contains a comma separated listing of value expressions, each of these optionally followed by ASC or DESC to indicate ascending (the default) or descending order. The expressions listed here have no connection with those in the select-part, i.e. it does not have to be subset of those.

"null" values are put at the end (when using ASC order).

4.1.4.2.12 The limit part

This can be used to limit the number of results returned from a query. This part is optional.

4.1.4.2.13 The option part

The option part allows to specify options that influence the execution of the query. The options are defined as:

option_name = 'option_value' (, option_name = 'option_value')*

Supported options:

name

value

default

include_retired

true/false

false

search_last_version

true/false

false

style_hint

(anything)

(empty)

annotate_link_fields

true/false

true

include_retired is used to indicate that retired documents should be included in the result (by default they are not).

search_last_version is used to indicate that the last version of metadata should be searched and retrieved, instead of the live version. When using this, documents that do not have a live version will also be included in the query result (otherwise they are not included). Full text searches are always performed on the live data, regardless of whether this option is specified.

style_hint is used to supply a hint to the publishing layer for how the result of the query should be styled. The repository server does not do anything more then add the value of this option as an attribute on the generated XML query results (<searchResult styleHint="my hint" ...). It is then up to the publishing layer to pick this up and do something useful with it. For how this is handled in the DaisyWiki, see the page on Query Styling.

annotate_link_fields indicates whether selected fields of type "link" should be annotated with the document name of the document pointed to by the link. If you don't need this, you can disable this to gain some performance.

4.1.4.3 Example queries

4.1.4.3.1 List of all documents
select id, name where true
4.1.4.3.2 Search on document name
select id, name where name like 'p%' order by creationTime desc limit 10
4.1.4.3.3 Show the 10 largest documents
select id, name, totalSizeOfParts where true order by totalSizeOfParts desc limit 10
4.1.4.3.4 Show documents of which the last version has not yet been published
select id, name, versionState, versionCreationTime
  where versionState = 'draft' option search_last_version = 'true'
4.1.4.3.5 Overview of all locks
select id, name, lockType, lockOwnerName, lockTimeAcquired, lockDuration
  where lockType is not null
4.1.4.3.6 All documents having a part containing an image
select id, name where HasPartWithMimeType('image/%')
4.1.4.3.7 Order documents randomly
select name where true order by Random()
4.1.4.3.8 Documents ordered by length of their name
select name, Length(name) where true order by Length(name) DESC

4.1.5 Full Text Indexer

Full text indexing in Daisy happens automatically when document variants are updated, so you do not need to worry about updating the index yourself. Technically, the full text indexer has a durable subscription on the JMS events generated by the repository, and it are these events which trigger the index updating.

4.1.5.1 Technology

Daisy uses Jakarta Lucene as full-text indexer.

4.1.5.2 Included content

Only document variants which have a live version are included in the full text index. Thus retired document variants or document variants having only draft versions are not included. It is the content of the live version which is indexed, thus full text search operations always search on the live content.

For each document variant, the included content consists of the document name, the value of string fields, and text extracted from the parts. For the parts, text extraction will be performed on the data if the mime type is one of the following:

Mime type

Comment

text/plain

text/xml

e.g. the "Daisy HTML" parts

application/xhtml+xml

XHTML documents

application/pdf

PDF files

application/vnd.sun.xml.writer

OpenOffice Writer files

application/msword
application/vnd.ms-word

Microsoft Word files

application/mspowerpoint
application/vnd.ms-powerpoint

Microsoft Powerpoint files

application/msexcel
application/vnd.ms-excel

Microsoft Excel files

Support for other formats can be added by implementing a simple interface. Ask on the Daisy Mailing List if you need more information about this.

4.1.5.3 Index management

It is possible to trigger optimisation of the index, and rebuilding of the index, through the JMX management interface (accessible through your web browser, runs by default on port 9264). Rebuilding the index can be useful for example when a new version of Daisy has support for new data formats. When rebuilding the index you can select the documents to be re-indexed using a query, or simply re-index all documents. For example, to re-index all PDF files you could enter the query "select id where HasPartWithMimeType('application/pdf')" (what you put in the select-part does not matter).

4.1.6 User Management

All operations done on the Daisy Repository Server are done as a certain user acting in a certain role(s). For this purpose, the Repository Server has a user management module to define the users and the roles. The authentication of the users is done by a separate component, allowing to plug in custom authentication techniques.

4.1.6.1 User Management

Users and roles are uniquely and permanently identified by a numeric ID, but they also have respectively a unique login and unique name.

A user has one or more roles. After logging in, it is both possible to have just one role active and let the user manually switch between his/her roles, or to have all roles of a user active at the same time (which is the behaviour traditionally associated with user groups). If a user has a default role, this role will be active after login. If no default role for the user is specified, all its roles will become active after login, with the exception of the Administrator role (if the user would have this role). This is because the Administrator role allows to do everything, which would then defeat the purpose of having other roles. If the user only has the Administrator role, then obviously that one will become active after login.

Users have a boolean flag called updateable by user: this indicates whether a user can update his/her own record. If true, a user can change its first name, last name, email and password. Role membership can of course not be changed, and neither can the login. It is useful to set this off for "shared users", for example the guest user in the Daisy Wiki application.

The Confirmed and ConfirmKey fields are used to support the well-known email-based verification mechanism in case of self-registration. If the Confirmed flag is false a user will not be able to log in.

4.1.6.1.1 The Administrator role

The repository server has one predefined role: Administrator (ID: 1). People having the role of Administrator as active role have a whole bunch of special privileges:

4.1.6.1.2 Predefined users and roles
4.1.6.1.2.1 $system

$system is a bootstrap user internally needed in the repository. The user $system cannot log in, so its password is irrelevant. This user should not (and cannot) be deleted, nor should it be renamed. Simply don't worry about it.

4.1.6.1.2.2 internal

The user "internal" is a user created during the initialisation of the Daisy repository. The user is used by various components that run inside the repository server to talk to the repository. By default, we also use this user in the repository client component that runs inside Cocoon, which needs a user to update its caches.

The internal user has (and should have) the Administrator role.

During installation, this user gets assigned a long random generated password (you can see it in the myconfig.xml or cocoon.xconf).

4.1.6.1.2.3 guest user and guest role (Daisy Wiki)

The Daisy Wiki predefines a user called guest and a role called guest. This user has the password "guest". This is the user that becomes automatically active when surfing to the Daisy Wiki application, without needing to log in. After initialisation of the Daisy Wiki, the ACL is configured to disallow any write operations for users having the guest role.

4.1.6.1.2.4 registrar (Daisy Wiki)

The registrar user is the user that will:

During installation, this user gets assigned a long random generated password (you can see it in the cocoon.xconf).

4.1.6.2 Authentication Schemes

Daisy provides its own password authentication, but it is also possible to delegate the authentication to an external system. At the time of this writing, Daisy ships with support for authentication using LDAP and NTLM. It is possible to configure multiple authentication schemes and to have different users authenticated against different authentication schemes.

The authentication schemes are configured in the myconfig.xml file (which is located in <daisy-data-dir>/conf). Just search on "ldap" or "ntlm" and you'll see the appropriate sections. After making changes there, you will need to restart the repository server. To let users use the newly defined authentication scheme(s), you need to edit their settings via the user editor on the administration pages.

Daisy does not do automatic synchronisation of user information (such as updating the e-mail address based on what is stored in LDAP), but it is possible to auto-create users on first log in. This means that when a user logs in for the first time in Daisy, and does not yet exist in Daisy, an authentication scheme is given the possibility to create the user (if it exist in the external system). To enable this feature, search in the myconfig.xml file for "authenticationSchemeForUserCreation".

To debug authentication problems, look at the log files in <daisy-data-dir>/logs/daisy-request-errors-<date>.log. Problems in the configuration of the authentication schemes do not ripple through over the HTTP interface of the repository, thus are not visible in the Daisy Wiki.

4.1.6.2.1 Implementing new authentication schemes

Implementing a new authentication scheme is done by:

For an example, you can look at the sources of the LDAP or NTLM authentication. The NTLM authentication in particular is splitted of from the main repository server sources and can be found in the directory services/ntlm-auth.

To create a new authentication scheme, you do not need to recompile Daisy from source. You will need the following jars in the classpath to compile the your new authentication scheme classes:

avalon-framework-api-<version>.jar
daisy-repository-api-<version>.jar
diasy-repository-server-spi-<version>.jar

4.1.7 Access Control

4.1.7.1 Introduction

This document explains Daisy's features for access control: the authorisation of document operations such as read and write.

While we usually talk about documents, technically the access control happens on the document variant level: a user is granted or denied access to a certain document variant.

In many systems, access control is configured by having access control lists (ACLs) attached to documents. These ACLs contain access control rules which tell for a certain users or roles (groups) what operations they can or cannot perform.

For Daisy, it was considered to be too laborious to manage ACLs for each individual document. Therefore, there is one global ACL, where you can select sets of documents based on an expression and then define the access control rules that apply to these documents.

4.1.7.2 Structure of the ACL

The structure of the ACL is illustrated by the diagram below.

In ACL terminology, an object is the protected resource, and a subject is an entity wanting to perform an operation on the object. The objects in our case are documents, selected using an expression. The subjects are users, which can be living organisms, usually humans, or programs acting on behalves of them.

As will become clear when reading about the evaluation of the ACL below, the order of the entries in the ACL is important.

4.1.7.2.1 Object specification

The expression used to select documents in the object specification uses the same syntax as in the where clause of an expression in the Daisy Query Language. However, the number of identifiers that are available is severely limited. More specifically, you can test on the following things:

Some examples of expressions:

InCollection('mycollection')

documentType = 'Navigation' and InCollection('mycollection')

$myfield = 'x' or $myotherfield = 'y'

For the evaluation of these expression, the data of the fields in the last version is used, not the data from the live version.

4.1.7.2.2 Access Control Entry

See diagram.

If the subject type is everyone, the subject value should be set to -1.

If you give 'read live' rights to someone, they are able to:

If you give 'read' rights to someone, they have full read access to the document (thus they can view all versions and the list of versions).

If you give 'write' rights to someone, they are able to:

The 'delete' right gives users the possibility to delete documents or document variants.

If you give 'publish' rights to someone, they are able to change the publish/draft state of versions of documents.

4.1.7.2.3 Staging and Live ACL

In Daisy, there are two ACLs: a staging ACL and a live ACL. Only the staging ACL is directly editable. So it is required to first edit the staging ACL, and then put it live (= copying the staging ACL over the live ACL). It is possible to first test the staging ACL: you can give a document id, a role id and a user id and get the result of ACL evaluation in return, including an explanation of which ACL rules made the final decision. In the Daisy Wiki front end, all these operations are available from the administration screens. It is recommended that after editing the ACL, you first test it before putting it live, so that you are sure there are no syntax errors in the document selection expressions.

4.1.7.3 Evaluation of the ACL: how is determined if someone gets access to a document

The determination of the authorisation of the various operations for a certain document happens as follows:

Further notes:

4.1.7.4 Other security aspects

This document only discussed authorisation of operations on documents for legitimate users. Other aspects of security include:

4.1.8 Email Notifier

4.1.8.1 General

Daisy can send out emails when changes are made to documents. To make use of this the SMTP host must be correctly configured, which is usually done as part of the installation, but can be changed afterwards (see below). In the Daisy Wiki, individual users can subscribe to get notifications by selecting the "User Settings" link, making sure their email address is filled in, and checking the checkbox next to "Receive email notifications of document-related changes.".

Users will only receive events of documents to which they have at least read (not 'read live') access rights. It is possible to receive notifications for individual documents, for all documents belonging to a certain collection, or for all documents. The mails will notify document creation, document updates or version state changes.

While we usually talk about documents, the actual notifications happen on the document variant level.

As you can see on the User Settings page, it is also possible to subscribe to other events: user, schema, collection and ACL related changes. However, for these events proper formatting of the mails is not yet implemented, they simply contain an XML dump of the event.

4.1.8.2 Configuration

Configuration of the email options happens in the <DAISY_DATA>/conf/myconfig.xml file. There you can configure:

After making any changes to the myconfig.xml file, the repository server needs to be restarted.

4.1.8.3 Implementation notes

The email notifier is an extension component running inside the repository server. It is independent of the Daisy Wiki. The email notifier provides a Java API for managing the subscriptions, as well as additions to the HTTP+XML interface (logical, because that's how the implementation of the Java API talks to the repository).

4.1.9 Document Task Manager

The purpose of the Document Task Manager (DTM) is to perform a certain task across a set of documents. The DTM is an optional component running inside the Daisy Repository Server. Some of its features are:

Tasks are executed in the background, inside the repository server. Thus the user (a person or another application) starting the task does not have to wait until it is completed, but can do something else and check later if the task ended successfully.

The execution progress of the task is maintained persistently in the database. For each document on which the task needs to be executed, you can consult whether it has been performed successfully, whether it failed (and why), or whether it still has to be executed. Since this information is tracked persistently in the database, it is not lost in case the server would be interrupted.

Tasks can be interrupted. Since the task is performed on one document after another, it is easily possible to interrupt between two documents.

Tasks can be written in Javascript or be composed from built-in actions. Executing custom Javascript-based tasks is only allowed by Administrators, since there is a certain risk associated with it. For example, it is possible to write a task containing an endless loop which would only be interruptible by shutting down the repository server, or a task could call System.exit() to shut down the server.

The execution details of a task, which are stored in the database, are cleaned up automatically after two weeks (by default), and can of course also be deleted manually.

The DTM is accessible via the HTTP API and the Java API.

The Daisy Wiki contains a frontend for starting new tasks and consulting the execution details of existing tasks.

Ideas for the future:

4.1.10 Publisher

4.1.10.1 Introduction

The publisher is an optional component that runs inside the repository server. Its goal is to retrieve in one remote call the information you want to display on a page. The result is returned as an XML document. The requestable information consists of more than just XML dumps of repository entities, the publisher provides all sorts of extra functionality, such as 'prepared documents' for publishing and diffs between document versions.

A publisher-request takes the form of an XML document which describes the various stuff you want the publisher to return. The remainder of this document describes the format of these XML requests.

The publisher was developed to support the needs of the Daisy Wiki, but can be useful for other applications as well.

4.1.10.2 The publisher request format

This section describes the format of the publisher requests. The responses are not described in much detail, they are easily viewable by trying the requests out on a live Daisy instance. This can, for example, be done by creating a file containing a publisher request and using a tool like wget to send it to the HTTP interface of the publisher, thus like this:

wget --post-file=pubreq.xml --http-user=testuser@1
     --http-passwd=testuser http://localhost:9263/publisher/request
4.1.10.2.1 p:publisherRequest

A basic, empty publisher request is structured as follows:

<p:publisherRequest
    xmlns:p="http://outerx.org/daisy/1.0#publisher"
    locale="en-US"
    versionMode="live"
    exceptions="throw">

 [... various publisher requests ...]

</p:publisherRequest>

The reponse of a publisher request is structured as follows:

<p:publisherResponse
    xmlns:p="http://outerx.org/daisy/1.0#publisher">

 [... responses to the various requests ...]

</p:publisherResponse>

About the attributes on the p:publisherRequest element:

The locale attribute is optional, by default the en-US locale will be used. If the publisher request is executed as part of another publisher request (see p:preparedDocuments), the locale will default to the locale of the 'parent' publisher request.

The versionMode attribute is optional, valid values are live (the default) or last. This attribute indicates which version of a document should be used by default if no explicit version is indicated. If its value is 'last', it will also cause the option 'search_last_version' to be set for various queries (those embedded in documents when requesting a preparedDocument, those executed by p:performQuery or p:forEach). It will also influence the version mode of the navigation tree when using p:navigationTree.

The exceptions attribute is also optional, throw is its default value. Basically this means that if an exception occurs during the processing of the publisher request, it will be thrown. It is also possible to specify inline, in which case the error description will be embedded in the p:publisherResponse element, but no exception will be thrown.

The styleHint attribute (optional, not shown above) will simply be replicated on the p:publisherResponse element. The publisher itself does not interpret the value of this attribute. See other information on document styling for when this is useful.

4.1.10.2.2 p:document

A p:document request is used to change the context document. The context document is the document on which the document related requests listed in the next section apply.

Any publisher request element can be nested within p:document.

The p:document request can work in three ways

4.1.10.2.2.1 Specify the document explicitly

Using the attributes id, branch (optional), language (optional) and version (optional) the new context document is specified.

The branch and language can be specified either by name or ID. If not specified, they default to the main branch and default language.

The version can be specified as "live" (the default), "last" or an explicit version number.

4.1.10.2.2.2 Specify a field attribute

When this p:document is used in a location where there is already a context document (e.g. from a parent p:document), it is possible to use an attribute called field. The value of this attribute should be the name of a link-type field. This p:document request will then change the context document to the document specified in that link-type field in the current context document. If the current context document does not have such a field, the p:document request will be silently skipped. If the link-type field is a multivalue field, the p:document request will be executed once for each value of the multivalue field. This will then lead to multiple sibling p:document elements in the publisher response.

4.1.10.2.2.3 Implicit

When the p:document request is used as child of the p:forEach request (see later), the context document is determined by the current document of the for-each loop.

4.1.10.2.2.4 Output of p:document

The p:document request will output a p:document element in the publisherResponse. This element will have attributes describing the exact document variant and version that the context document was changed to.

4.1.10.2.3 Requests that can only be used when a context document is available
4.1.10.2.3.1 p:aclInfo

Evaluates the document against the ACL and returns the result. This request requires no attributes.

4.1.10.2.3.2 p:subscriptionInfo

Returns whether the user is subscribed for email notifications for this document, the result is a tag like this:

<p:subscriptionInfo subscribed="true|false"/>

This request requires no attributes.

4.1.10.2.3.3 p:comments

Returns the comments for the current document. Newlines in the comments are replaced with <br/> tags. This request requires no attributes.

4.1.10.2.3.4 p:availableVariants

Returns the available variants for this document. This request requires no attributes.

4.1.10.2.3.5 p:preparedDocuments (& p:prepareDocument)

p:preparedDocuments is the most important of all publisher requests. It returns the content of the document prepared for publishing. The preparation consists of all sorts of things such as:

The full format of the request is:

<p:preparedDocuments applyDocumentTypeStyling="true|false" publisherRequestSet="...">
  <p:navigationDocument id="..." branch="..." language="..."/>
</p:preparedDocuments>

The applyDocumentTypeStyling attribute isn't used by the publisher, but is simply replicated in the result. In the Daisy Wiki, its purpose is to indicate whether document type specific styling should be automatically applied.

The publisherRequestSet attribute: see below.

The p:navigationDocument element is optional. If supplied, it enables to annotate "daisy:" links with the target path in the navigation tree.

4.1.10.2.3.5.1 How it works

p:preparedDocuments looks up a new publisher request to be performed on the context document. The publisher request to be used can be determined dynamically (described further on), but by default it is this one:

<p:publisherRequest xmlns:p="http://outerx.org/daisy/1.0#publisher">
  <p:prepareDocument/>
  <p:aclInfo/>
  <p:subscriptionInfo/>
</p:publisherRequest>

This publisher request should (usually) contain a <p:prepareDocument/> element. The p:prepareDocument will be replaced by the Daisy document as XML (d:document), in which the HTML content is inlined and processed (i.e. the things mentioned in the enumeration above). If the content contains an include of another document, then for this included document the publisher will again determine a publisher request to be performed upon it, and execute it. The same happens for each include (recursively). The results of all these publisher requests are inserted in the publisher response in a structure like this:

<p:preparedDocuments applyDocumentTypeSpecificStyling="true|false">

  <p:preparedDocument id="1">
    <p:publisherResponse>
      <d:document ...
    </p:publisherResponse>
  </p:preparedDocument>

  <p:preparedDocument id="2">
    <p:publisherResponse>
      <d:document ...
    </p:publisherResponse>
  </p:preparedDocument>

</p:preparedDocuments>

The publisher response of the context document will always end up in the p:preparedDocument element with attribute id="1".  If the document includes no other documents, this will be the only p:preparedDocument. Otherwise, for each included document (directly or indirectly), an additional p:preparedDocument element will be present.

So the included documents are not returned in a nested structure, but as a flat list. This allows to perform custom styling on each separate document before nesting them.

On the actual location of an include, a p:daisyPreparedInclude element is inserted, with an id attribute referencing the related p:preparedDocument element.

The content of a p:preparedDocument element is thus a single p:publisherResponse element, which in turns contains a single d:document element (as result of the p:prepareDocument in the publisher request). This d:document element follows the standard form of XML as is otherwise retrieved via the HTTP interface or by using the getXml() method on a document object, but with lots of additions such as inlined content for Daisy-HTML parts and non-string attribute values formatted according to the specified locale.

If you requested the live version of the document, but the document does not have a live version, there will simply be no p:preparedDocuments element in the response.

4.1.10.2.3.5.2 Determination of the publisher request to be performed

If instead of the default publisher request mentioned above, you want to execute some custom publisher request (which can be used to retrieve information related to the document being published), then this is possible by defining a publisher request set.

In the data directory of the repository server, you will find a subdirectory called 'pubreqs'. In this directory, each subdirectory specifies a publisher request set. Each such subdirectory should contain a file called mapping.xml and one or more other files containing publisher requests.

The mapping.xml file looks like this:

<m:publisherMapping xmlns:m="http://outerx.org/daisy/1.0#publishermapping">
  <m:when test="documentType = 'mydoctype'" use="myrequest.xml"/>
  <m:when test="true" use="default.xml"/>
</m:publisherMapping>

The publisher will run over each of the when rules, and if the expression in the test attribute matches, it will use the publisher request specified in the use attribute. The expressions are the same as used in the query language, and thus also the same as used in the ACL definition.

To make use of such a specific set of publisher requests, you use the publisherRequestSet attribute on the p:preparedDocuments element. The value of this attribute should correspond to the name of subdirectory of the pubreqs directory.

In the Daisy Wiki, the publisher request set to be used can be specified in the siteconf.xml

4.1.10.2.3.5.3 p:prepareDocument

The p:prepareDocument can have an optional attribute called inlineParts. This attribute specifies a comma-separated list of part type names (or IDs) for which the content should be inlined. By default this only happens for parts for which the Daisy-HTML flag is set to true.

The inlining will only happen if the actual part has a mime-type that starts with "text/" or if the mime-type is recognized as an XML mime-type. Recognized XML mime-types are currently text/xml, application/xml or any mime type ending with "+xml".

If it is an XML mime-type, then the content will be parsed and inserted as XML. Otherwise, it will be inserted as character data (assuming UTF-8 encoding of the part text data). If the inlining actually happened, an attribute inlined="true" is added to the d:part element in question.

4.1.10.2.3.6 p:shallowAnnotatedVersion

Returns the shallow version XML (this the version XML without field and part information in it), thus a "d:version" element. This request requires no attributes.

If you requested the live version of the document, but the document does not have a live version, there will simply be no d:version element in the response.

4.1.10.2.3.7 p:annotatedDocument

Returns the document XML with some slight annotations, thus a d:document element. This request requires no attributes.

In contrast with most other elements, if you request the live version of the document but the document doesn't have a live version, this element will automatically fallback to the last version, so there will always be a d:document element in the response.

4.1.10.2.3.8 p:annotatedVersionList

Returns a list of all versions of the document (as a d:versions element). This request requires no attributes.

4.1.10.2.3.9 p:diff

Returns a diff of the document version with another version of this document or another document.

The full form of this request is:

<p:diff>
  <p:otherDocument id="..."  branch="..." language="..." version="..."/>
</p:diff>

If no p:otherDocument element is specified, the diff will automatically be taken with the previous version of the document. If there is no such version (because the document has only one version yet), there will be no diff response.

If a p:otherDocument element is supplied, any combination of attributes is allowed, in other words all attributes are optional.

4.1.10.2.4 p:navigationTree

Returns a navigation tree. The full form of this request is:

<p:navigationTree>
  <p:navigationDocument id="..." branch="..." language="..."/>
  <p:activeDocument id="..." branch="..." language="..."/>
  <p:activePath>...</p:activePath>
  <p:contextualized>true|false</p:contexutalized>
  <p:versionMode>live|last</p:versionMode>
</p:navigationTree>

The activeDocument, activePath and versionMode elements are optional.

4.1.10.2.5 p:myComments

Returns a list of all private comments of the user.

4.1.10.2.6 p:performQuery

Returns the result of executing a query. The full form of this request is: 

<p:performQuery>
  <p:query>...</p:query>
  <p:extraConditions>...</p:extraConditions>
</p:performQuery>

The p:extraConditions element is optional, and specifies additional conditions that will be AND-ed with those in the where clause of the query. This feature is not often needed. It can be useful when you let the user enter a free query but want to enforce some condition, e.g. limit the documents to a certain collection.

The result of a query is a d:searchResult element.

If there is a context document available (i.e. if this p:performQuery is used inside a p:document) then the ContextDoc function can be used in the query.

4.1.10.2.7 p:forEach

The p:forEach element allows to perform a p:document request for each document (actually, document variant) returned by query. The full form of this request is:

<p:forEach useLastVersion="true|false">
  <p:query>...</p:query>
  <p:document .... />
</p:forEach>

If the useLastVersion attribute is false or not specified, the live version of each document will be used, otherwise the last version.

If there is a context document available (i.e. if this p:performQuery is used inside a p:document) then the ContextDoc function can be used in the query.

4.1.10.2.8 p:if

Allows to execute a part of the publisher request only if a certain test is satisfied. Syntax:

<p:if test="...">
  [... any publisher request ...]
</p:if>

The test attribute specifies a conditional expression (an expression evaluating to true or false) in the same format as used in the Daisy query language.

4.1.10.2.9 p:choose
<p:choose>
  <p:when test="...">
     [... any publisher request ...]
  </p:when>

  [... more p:when's ...]

  <p:otherwise>
    [... any publisher request ...]
  </p:otherwise>
</p:choose>

There should be at least one p:when, the p:otherwise is optional.

4.1.10.2.10 p:group

The p:group element performs no function by itself, except to act as a container for other requests. It needs an id attribute:

<p:group id="...">

 [... other requests here ...]

</p:group>

It allows to distinguish between e.g. different queries or navigation tree results if you would have more then one of them.

4.1.10.2.11 p:resolveDocumentIds

This element allows to retrieve the names of a set of documents of which you have only the ID. The advantage compared to using simply the repository API is that this only requires one remote call for as many documents as you need (assuming you are using the remote API, otherwise it does not make a difference).

<p:resolveDocumentIds branch="..." language="...">
  <p:document id="..." branch="..." language="..." version="..."/>
</p:resolveDocumentIds>

The branch and language attributes on the p:resolveDocumentIds element specify the default branch and language to use if it is not specified on the individual documents. These attributes are optional, the main branch and default language is used as default when these attributes are not specified.

The branch, language and version attributes on the p:document element are optional. By default the live version is used, or the last version if the document does not have a live version.

The result has the following format:

<p:resolvedDocumentIds>
  <p:document id="..." branch="..." language="..." version="..." name="..."/>
</p:resolvedDocumentIds>

The id, branch, language and version attributes are simply copied from the requesting document element. The name attribute is added. The p:document elements in the result corresponds to those in the request at the same position.

If there is some error (such as the document does not exist, or the specified branch or language does not exist), an error message is put in the name attribute.

4.1.11 Backup locking

The practical side of making backups is explained in the section Making backups. Here we only describe the backup-lock mechanism, which is of use if you want to write your own backup tool.

The repository server uses multiple storages: a relational database, the blobstore (a filesystem directory), and the full text indexes (also stored on the filesystem). Next to that, the JMS system also has its own database. Daisy does not employ some fancy distributed transaction manager with associated log, therefore it has its own simple mechanism to allow to take a consistent backup of these different stores. Daisy allows to take a "backup lock" on the repository server. This will:

A backup lock is requested via the JMX interface. It needs a "timeout" parameter that specifies how long to wait for any running operations to end.

While the backup lock is active, all read operations will continue to work, and update operations that only involve the repository's relational database will continue to work. Operations that require an update to the blobstore (such as saving a document in most cases) will give an exception.

4.1.12 Image thumbnails and metadata extraction

The repository server contains an (optional) component that can perform image thumbnailing and extraction of metadata (width, height and, for jpeg, arbitrary EXIF fields). This component is registered as a "pre-save-hook", this is a component which gets called before a document is saved, and which can modify the content of the to-be-saved document.

For further reference, we will call this component the image hook.

The image hook can be configured to react on multiple document types, and for each document type it is possible to specify what needs to be done:

By default, the image hook is configured to handle the Image document type as defined by the Daisy Wiki.

The configuration of the image hook can be adjusted via the myconfig.xml configuration file of the repository server.

4.2 Programming interfaces

4.2.1 Scripting the repository using Javascript

4.2.1.1 Introduction

Rhino, a Java-based Javascript implementation, makes it easy to use the Java API of the repository server to automate all kinds of operations. In other words: easy scripting of the repository server. It brings all the benefits of Daisy's high-level repository API without requiring Java knowledge or the setup of a development environment.

4.2.1.2 How does it work?

  1. Write a Javascript, save it in a ".js" file.
  2. Open a command prompt or shell, set the DAISY_HOME environment variable to point to your Daisy installation
  3. Go to the directory <DAISY_HOME>/bin
  4. Execute "daisy-js <name-of-scriptfile>"

4.2.1.3 Connecting to the repository server from Javascript

The basic code you need to connect to the repository server from Javascript is the following:

importPackage(Packages.org.outerj.daisy.repository);
importClass(Packages.org.outerj.daisy.repository.clientimpl.RemoteRepositoryManager);

var repositoryManager = new RemoteRepositoryManager("http://localhost:9263",
                                                    new Credentials("testuser", "testuser"));
var repository = repositoryManager.getRepository(new Credentials("testuser", "testuser"));

Some explanation:

The importPackage and importClass statements are used to make the Daisy Java API available in the Javascript environment.

Then a RepositoryManager is constructed, this is an object from which Repository objects can be retrieved. A Repository object represents a connection to the Daisy Repository Server for a certain user. Typically, you only construct one RepositoryManager, and then retrieve different Repository objects from it if you want to perform actions under different users.

The first argument of the RepositoryManager constructor is the address of the HTTP interface of the repository server (9263 is the default port). The second argument is a username and password for a user that is used inside the implementation to fill up caches. Currently, this user has to be a user which has the Administrator role. (Inside the implementation, some often needed info like the repository schema and the collections is cached)

Then from the RepositoryManager a Repository for a specific user is retrieved. Here we use the same credentials as for the cache user of the RepositoryManager, but this doesn't have to be the case.

4.2.1.4 Repository Java API documentation

Reference documentation of the Daisy API is included in the binary distribution in the apidocs directory (open the file index.html in a web browser). See also Java API.

4.2.1.5 Examples

4.2.1.5.1 Creating a document (uploading an image)

This example uploads an image called "myimage.gif" from the current directory into the repository.

importPackage(Packages.org.outerj.daisy.repository);
importClass(Packages.org.outerj.daisy.repository.clientimpl.RemoteRepositoryManager);

var repositoryManager = new RemoteRepositoryManager("http://localhost:9263",
                                                    new Credentials("testuser", "testuser"));
var repository = repositoryManager.getRepository(new Credentials("testuser", "testuser"));

var document = repository.createDocument("My test image", "Image");
var imageFile = new java.io.File("myimage.gif");
document.setPart("ImageData", "image/gif", new FilePartDataSource(imageFile));
document.save();

print("Document created, ID = " + document.getId());

See the API documentation for the purpose of the arguments of the methods. For example, the text "Image" supplied as the second argument of the createDocument method is the name of the document type to use for the document. Likewise, the first argument of setPart, "ImageData", is the name of the part.

It would be an interesting exercise to extend this example to upload a whole directory of images :-)

4.2.1.5.2 Performing a query
importPackage(Packages.org.outerj.daisy.repository);
importClass(Packages.org.outerj.daisy.repository.clientimpl.RemoteRepositoryManager);
importPackage(Packages.java.util);

var repositoryManager = new RemoteRepositoryManager("http://localhost:9263",
                                                    new Credentials("testuser", "testuser"));
var repository = repositoryManager.getRepository(new Credentials("testuser", "testuser"));
var queryManager = repository.getQueryManager();

var searchresults = queryManager.performQuery("select id, name where true", Locale.getDefault());
var rows = searchresults.getSearchResult().getRows().getRowArray();
for (var i = 0; i < rows.length; i++) {
  print(rows[i].getValueArray(0) + " : " + rows[i].getValueArray(1));
}

print("Total number: " + rows.length);
4.2.1.5.3 Creating a user
importPackage(Packages.org.outerj.daisy.repository);
importClass(Packages.org.outerj.daisy.repository.clientimpl.RemoteRepositoryManager);

var repositoryManager = new RemoteRepositoryManager("http://localhost:9263",
                                                    new Credentials("testuser", "testuser"));
var repository = repositoryManager.getRepository(new Credentials("testuser", "testuser"));

// With the UserManager we can manage users and roles
var userManager = repository.getUserManager();

// Get references to some roles, we'll need them in a moment
var guestRole = userManager.getRole("guest", false);
var adminRole = userManager.getRole("Administrator", false);

// check if current user has admin role, and exit if not
var me  = userManager.getUser(repository.getUserId(), false);
var adminRoleId = adminRole.getId();
if (!me.hasRole(adminRoleId))
{
  print ("current user lacks admin rights to add new user. ");
  exit;
}
repository.switchRole(adminRoleId);

// Create the new user
var newUser = userManager.createUser("john");

// The user needs to be added to at least one role
newUser.addToRole(guestRole);
newUser.addToRole(adminRole);

// Optionally, set a default role which will be active after
// logging in. If not set, all roles (with the exception of
// the Administrator role) will be active on login
// newUser.setDefaultRole(guestRole);

// Password is required when using Daisy's built-in authentication scheme
newUser.setPassword("boe");

// Alternatively, set another authentication scheme:
// newUser.setAuthenticationScheme("my-ldap");

// Optional things
newUser.setFirstName("John");
newUser.setLastName("Johnson");

// Setting updateableByUser will allow the user to access the
// user settings page in the Wiki, so that the user can
// update here e-mail
newUser.setUpdateableByUser(true);

newUser.save();
4.2.1.5.4 Your example here

If you've got a cool example to contribute, just write to the Daisy mailing list.

4.2.2 Java API

4.2.2.1 Introduction

Daisy is written in Java and thus its native interface is a Java API. This Java API is packaged separately, and consists of two jars:

daisy-repository-api-<version>.jar
daisy-repository-xmlschema-bindings-<version>.jar

The second jar, the xmlschema-bindings, are Java classes generated from XML Schemas, and form a part of the API. To write client code that talks to Daisy, at compile you need only the above two jars in the classpath (at runtime, you need a concrete implementation, see further on).

There are two implementations of this API available:

This is illustrated in the diagram below.

To be workable, the remote implementation caches certain information: the repository schema (document, field and part types), the collections, and the users (needed to be able to quickly map user IDs to user names). To be aware of changes done by other clients, the remote implementation can listen to the JMS events broadcasted by the server to update these caches. This is optional, for example a short-running client application that performs a specific task probably doesn't care much about this, especially since the cached information is not the kind of information that changes frequently. Even when JMS-based cache invalidation is disabled, the caches of a certain remote implementation instance are of course kept up-to-date for changes done through that specific instance.

Examples of applications making use of the remote API implementation are the Daisy Wiki, and the installation utilities daisy-wiki-init and daisy-wiki-add-site. Especially the source of those last two can serve as useful but simple examples of how to write client applications. The shell scripts to launch them show the required classpath libraries.

4.2.2.2 Quick introduction to the Java API

The Daisy Java API is quite high-level, and thus easy-to-use. The start point to do any work is the RepositoryManager interface, which has the following method:

Repository getRepository(Credentials credentials) throws RepositoryException;

The Credentials parameter is simply an object containing the user name and password. By calling the getRepository method, you get an instance of Repository, through which you can access all the repository functionality. The obtained Repository instance is specific for the user specified when calling getRepository. The Repository object does not need to be released after use. It is a quite lightweight object, mainly containing the authentication information.

Let's have a look at some of the methods of the Repository interface.

Document createDocument(String name, String documentTypeName);

Creates a new document with the given name, and using the named document type. The document is not immediately created in the repository, to do this you need to call the save() mehod on the Document. But first you need to set all required fields and parts, otherwise the save will fail (it is possible to circumvent this, see the full javadocs).

Document getDocument(long documentId, boolean updateable) throws RepositoryException;

Retrieves an existing document, specified by its ID. If the flag 'updateable' is false, the repository will return a read-only Document object, which allows it to return a shared cached copy. In the remote implementation, this doesn't matter since it doesn't perform any caching, but in the local implementation this can make a very huge difference.

RepositorySchema getRepositorySchema();

Returns an instance of RepositorySchema, through which you can inspect and modify the repository schema (these are the document, part and field types).

AccessManager getAccessManager();

Returns an instance of AccessManager, through which you can inspect and modify the ACL, and get the ACL evaluation result for a certain document-user-role combination.

QueryManager getQueryManager();

Returns an instance of QueryManager, through which you can perform queries on the repository using the Daisy Query Language.

CollectionManager getCollectionManager();

Returns an instance of CollectionManager, through which you can create, modify and delete document collections.

UserManager getUserManager();

Returns an instance of UserManager, through which you can create, modify and delete users.

The above was just to give a broad idea of the functionality available through the API. For more details, consult the complete JavaDoc of the API.

4.2.2.3 Writing a Java client application

Let's now look at a practical example.

Here's a list of jars you need in the CLASSPATH to use the remote repository API implementation.

This list is bound to change in different Daisy releases. We advise you to copy the settings of the CLASSPATH defined in the daisy-js (daisy-js.bat on Windows) script. (possibly removing the 'rhino' and 'daisy-javascript' jars)
The list below was last updated for Daisy 1.5.1:

DAISY_HOME/lib/daisy/jars/daisy-repository-api-1.5.jar
DAISY_HOME/lib/daisy/jars/daisy-repository-client-impl-1.5.jar
DAISY_HOME/lib/daisy/jars/daisy-repository-spi-1.5.jar
DAISY_HOME/lib/daisy/jars/daisy-util-1.5.jar
DAISY_HOME/lib/avalon-framework/jars/avalon-framework-api-4.1.5.jar
DAISY_HOME/lib/daisy/jars/daisy-repository-common-impl-1.5.jar
DAISY_HOME/lib/commons-httpclient/jars/commons-httpclient-2.0.2.jar
DAISY_HOME/lib/xmlbeans/jars/xbean-2.1.0.jar
DAISY_HOME/lib/xmlbeans/jars/xmlpublic-2.1.0.jar
DAISY_HOME/lib/stax/jars/stax-api-1.0.jar
DAISY_HOME/lib/daisy/jars/daisy-repository-xmlschema-bindings-1.5.jar
DAISY_HOME/lib/concurrent/jars/concurrent-1.3.2.jar
DAISY_HOME/lib/commons-logging/jars/commons-logging-1.0.4.jar
DAISY_HOME/lib/commons-collections/jars/commons-collections-3.1.jar
DAISY_HOME/lib/daisy/jars/daisy-jmsclient-api-1.5.jar
DAISY_HOME/lib/geronimo-spec/jars/geronimo-spec-jms-1.1-rc4.jar

(below only if you need the htmlcleaner)

DAISY_HOME/lib/daisy/jars/daisy-htmlcleaner-1.5.jar
DAISY_HOME/lib/nekohtml/jars/nekohtml-0.9.5.jar
DAISY_HOME/lib/nekodtd/jars/nekodtd-0.1.11.jar
DAISY_HOME/lib/xerces/jars/xercesImpl-2.6.2.jar
DAISY_HOME/lib/xerces/jars/xmlParserAPIs-2.2.1.jar

So depending on your own habits, you could set up a project in your IDE with these jars in the classpath, or make an Ant project, or whatever.

Below a simple and harmless example is shown: performing a query on the repository.

package mypackage;

import org.outerj.daisy.repository.RepositoryManager;
import org.outerj.daisy.repository.Credentials;
import org.outerj.daisy.repository.Repository;
import org.outerj.daisy.repository.query.QueryManager;
import org.outerj.daisy.repository.clientimpl.RemoteRepositoryManager;
import org.outerx.daisy.x10.SearchResultDocument;

import java.util.Locale;

public class Search {
    public static void main(String[] args) throws Exception {
        RepositoryManager repositoryManager = new RemoteRepositoryManager(
            "http://localhost:9263", new Credentials("testuser", "testuser"));
        Repository repository =
            repositoryManager.getRepository(new Credentials("testuser", "testuser"));
        QueryManager queryManager = repository.getQueryManager();

        SearchResultDocument searchresults =
            queryManager.performQuery("select id, name where true", Locale.getDefault());
        SearchResultDocument.SearchResult.Rows.Row[] rows =
            searchresults.getSearchResult().getRows().getRowArray();

        for (int i = 0; i < rows.length; i++) {
            String id = rows[i].getValueArray(0);
            String name = rows[i].getValueArray(1);
            System.out.println(id + " : " + name);
        }

        System.out.println("Total number: " + rows.length);

    }
}

The credentials supplied in the constructor of the RemoteRepositoryManager specify a user to be used for filling the caches in the repository client. This user currently needs to have the Administrator role.

4.2.2.4 Java client application with Cache Invalidation

Needs updating for switch to ActiveMQ (java sample code also need updating due to new required JMS client ID)

For long-running client applications you may want to have the caches of the client invalidated when changes happen by other users. For a code sample of how to create a JMS client and pass it on to the RemoteRepositoryManager, see JMS Cache Invalidation Sample.

For this example to run, you'll need the JMS client implementation jars in the CLASSPATH, in addition to the earlier listed jars:

DAISY_HOME/lib/daisy/jars/daisy-jmsclient-impl-1.3.jar
DAISY_HOME/lib/exolabcore/jars/exolabcore-0.3.7.jar
DAISY_HOME/lib/openjms/jars/openjms-client-0.7.6.jar

4.2.2.5 More

It might be interesting to also have a look at the notes on scripting using Javascript, since there essentially the same API is used from a different language.

4.2.3 HTTP API

4.2.3.1 Introduction

Daisy contains a HTTP+XML interface, which is an interface to talk to the repository server by exchanging XML messages over the HTTP protocol. This interface offers full access to all functionality of the repository.

The HTTP protocol is a protocol that allows to perform a limited number of methods (Daisy uses GET, POST and DELETE) on an unlimited number of resources, which are identified by URIs. The GET method is used to retrieve a representation of the addressed resource, POST to trigger a process that modifies the addressed resource, and DELETE to delete a resource.

With HTTP, all calls are independent of each other, there is no session with the server.

The Daisy HTTP interface listens by default on port 9263. You can easily try it out, for example if Daisy is running on your localhost, just enter the URL below in the location bar of the browser, and press enter. The browser will then send a GET request to the server. The example given here is a request to execute a query (written in the Daisy Query Language). This request doesn't require an XML payload, all parameters are specified as part of the URL. Note that spaces in an URL must be encoded with a plus symbol.

http://localhost:9263/repository/query?q=select+id,name+where+true&locale=en

The browser will ask a user name and password, enter your Daisy repository username and password (e.g., the one you otherwise use to log in on the Daisy Wiki), or use the user name "guest" and password "guest" (only works if you installed the Daisy Wiki). The browser will show the XML response received from the server (in some browsers, you might need to do "view source" to see it).

Not all operations can be performed as easily as the above example: some require POST or DELETE as method, some require an XML document in the body of the request, and some even require a multipart-formatted request body (the document create and update operations, which need to upload the binary part data next to the XML message). If you have a programming language with a decent HTTP client library, none of this should be a problem.

4.2.3.2 Authentication

All requests require authentication. Authentication is done using BASIC authentication.

If you want to log in as another role then the default role of a user, append "@<roleid>" to the login (without the quotes). Note that it must be the id of the role, not its name. For example, if your default role is not Administrator (ID: 1), but you would like to perform the request as Administrator, and your login is "jules", you would use "jules@1". When the login itself contains an @-symbol, it must be escaped by doubling it (i.e. each @ should be replaced with @@). Multiple active roles can be specified using a comma-separated list, e.g. "jules@1,105".

4.2.3.3 Robustness

The current implementation doesn't do (many) checks on the XMLs posted as part of a HTTP request. This means that for example missing elements or attributes might simply cause little-descriptive (but harmless) "NullPointerExceptions" to occur.

The reason for this is that we use the HTTP API mostly via the repository Java client, which generates valid messages for us.

Since the XML posted to a resource is usually the same as the XML retrieved via GET on the same resource, it is easy to get examples of correct XML messages. XML Schemas are also available (see further on), though being schema-valid doesn't necessarily imply the message is correct.

4.2.3.4 Error handling

If a response was handled correctly, the server will answer with HTTP status code 200 (OK). If the status code has another value, it means something went wrong.

For errors generated explicitly, or when a Java exception occurs, an XML message is created describing the exception, and is returned with a status code 202 (Accepted). The XML message consists of an <error> root element, with as child either a <description> element or a <cause> element. The <description> element contains a simple string describing the error. The <cause> element is used in case a Java exception was handled, and contains further elements describing the exception (including stacktrace), and can include <cause> elements recursively describing the "causing" exceptions of that exception. To see an example of this, simply do a request for a non-existing resource, e.g.:

http://localhost:9263/repository/document/99999999

(assuming there is no document with ID 99999999)

When executing a method (GET, POST, DELETE, ...) on a resource that doesn't support that method you will get status code 405 (Method Not Allowed).

Incorrect or missing authentication information will give status code 401 (Unauthorized).

Missing request parameters, or invalid ones (e.g. giving a string where a number was expected) will give status code 400 (Bad Request).

Doing a request for a non-existing resource will give status code 404 (Not Found)

4.2.3.5 Intro to the reference

The rest of this document describes the available URLs, the operations that can be performed upon them, and the format of the XML messages. The descriptions can be dense, the current goal of this document is just to give a broad overview, more details might be added later. You can always ask for more information on the Daisy Mailing List.

You can also investigate how things are supposed to work by monitoring the HTTP traffic between the Daisy Wiki and the Daisy Repository Server.

Sometimes XML Schema files are referenced, these can be found in the Daisy source distribution.

4.2.3.6 Core Repository Interface

4.2.3.6.1 Documents

On many document-related resources, request parameters called branch and language can be added (this will be mentioned in each case). The value of these parameters can be either a name or ID of a branch or language. If not specified, the branch "main" and the language "default" are assumed.

4.2.3.6.1.1 /repository/document

This resource represents the set of all documents. GET is not supported on this resource (you can retrieve a list of all documents using a query).

POST on this resource is used to create a new document, which also implies the creation of a document variant, since a document cannot exist without a document variant. The payload should be a multipart request having one multipartrequest-part (we use this long name to distinguish with Daisy's document parts) containing the XML description of the new document, and other multipartrequest-parts containing the content of the document parts (if any). The multipartrequest-part containing the XML should be called "xml", and should conform to the document.xsd schema. The part elements in the XML should have dataRef attributes whose value is the name of the multipartrequest-part containing the data for that part.

The server will return the XML description of the newly created document as result. This XML will, among other things, have the id attribute completed with the ID of the new document.

4.2.3.6.1.1.1 Example scenario: creating a new document

This example illustrates how to create a new document in the repository over the HTTP interface using the curl tool. Curl is a handy command-line tool to do HTTP (and other) requests, and is standard available on many Linux distributions (it exists for Windows too).

Suppose we want to create a new document of type 'SimpleDocument' (as used in the Daisy Wiki), with the part 'SimpleDocumentContent'. We start by creating the XML description of the document, and save it in a file called newdoc.xml:

<?xml version="1.0" encoding="UTF-8"?>
<ns:document
  xmlns:ns="http://outerx.org/daisy/1.0"
  name="My test doc"
  typeId="2"
  owner="3"
  validateOnSave="true"
  newVersionState="publish"
  retired="false"
  private="false"
  branchId="1"
  languageId="1"
  >
  <ns:customFields/>
  <ns:collectionIds>
    <ns:collectionId>1</ns:collectionId>
  </ns:collectionIds>
  <ns:fields/>
  <ns:parts>
    <ns:part mimeType="text/xml" typeId="2" dataRef="data1"/>
  </ns:parts>
  <ns:links/>
</ns:document>

Some items in the above XML will need to be changed for your installation:

Now we need to create a file containing the content of the part we're going to add. For example create a file called 'mynewfile.xml' and put the following in it:

<html>
  <body>

    <h1>Hi there!</h1>

    <p>This is a test document.</p>

  </body>
</html>

Finally we are ready to create the document using curl:

curl --basic
     --user testuser:testuser
     --form xml=@newdoc.xml
     --form data1=@mynewfile.xml
     http://localhost:9263/repository/document

You need to enter all arguments on one line of course, and change user, password and server URLs as appropriate for your installation. Note that the form parameter 'data1' corresponds to the dataRef attribute in the newdoc.xml file (you can choose any name you want for these, if you have multiple parts use different names)

4.2.3.6.1.2 /repository/document/<id>

<id> should be replaced with the ID of an existing document.

4.2.3.6.1.2.1 Retrieving a document

GET on this resource retrieves an XML description of a document, with a certain variant of the document. The XML will contain the data of the most recent version of the document variant. The (binary) part data is not embedded in the XML, but must be retrieved separately using the following URL (described further on):

/repository/document/<document-id>/version/<version-id>/part/<parttype-id>/data

To specify the document variant, add the optional request parameters branch and language.

4.2.3.6.1.2.2 Creating a document or adding a document variant

POST on this resource is used to update a document (and/or document variant), or to add a new variant to it. When adding a new variant there are two possibilities: initialise the new variant with the content of an existing variant, or create a new variant from scratch. We now describe these three distinct cases.

To update an existing document (document variant), the format of the POST is similar as when creating a document, that is, it should contain a multipart-format body. The XML in this case should be an updated copy of the XML retrieved via the GET on this resource. Unmodified parts don't need to be uploaded again.

To create a new variant from scratch, again the POST data is similar as when creating a new document. In addition, three request parameters must be specified:

Although the variant is created from scratch, it is only possible to add a new variant to a document if you have at least read access to an existing variant. The new variant to be created is specified by the branchId and languageId attributes within the posted XML.

Creating a new variant based on an existing variant is rather different. In this case no XML body or multipart-request must be done, but a POST operation with the following request parameters:

These parameter names explain themselves I think. The branches and languages can be specified either by name or ID.

4.2.3.6.1.2.3 Deleting a document or a document variant

DELETE on this resource permanently deletes the document. This will delete the document and all its variants.

To delete only one variant of the document, specify the request parameters branch and language.

4.2.3.6.1.3 /repository/document/<id>/version

GET on this resource returns a list of all versions in a document variant as XML. For each version, only some basic information is included (the things typically needed to show a version overview page).

To specify the document variant, add the optional request parameters branch and language.

4.2.3.6.1.4 /repository/document/<id>/version/<id>

GET on this resource returns the full XML description of this version. As when requesting a document, the actual binary part data is not embedded in the XML but has to be retrieved separately.

POST on this resource is used to modify the version state (which is the only thing of a version that can be modified, other then that, versions are read-only once created). The request should have two parameters:

For both the GET and POST methods, add the optional request parameters branch and language to specify the variant.

4.2.3.6.1.5 /repository/document/<id>/version/<id>/part/<id>/data

GET on this resource retrieves the data of a part in a certain version of a document. The meaning of the <id>'s is as follows:

  1. The first <id> is the document ID
  2. The second <id> the version ID (1, 2, 3, ...) or the string "last" to signify the last version
  3. The third <id> is the part type ID or the part type name of the part to be retrieved (if the first character is a digit, it is supposed to be an ID. Part type names cannot begin with a digit).

To specify the document variant, add the optional request parameters branch and language.

See also the /publisher/blob resource, which sets correct mime type, last modified and content length headers.

4.2.3.6.1.6 /repository/document/<id>/lock

See the lock.xsd file for the XML Schema of the XML used to interact with this resource.

GET on this resource returns information about the lock, if any.

POST on this resource is used to create a lock. In this case, all attributes in the XML must have a value except for the hasLock attribute.

DELETE on this resource is used to remove the lock (if any). No request body is required.

For all three methods, the returned result is the XML description of the lock after the performed operation (possibly describing that there is no lock).

A lock applies to a certain variant of a document. To specify the document variant, add the optional request parameters branch and language.

4.2.3.6.1.7 /repository/document/<id>/comment

See comment.xsd for the XML Schema of the messages.

GET on this resource returns the list of comments for a document variant. To specify the document variant, add the optional request parameters branch and language.

POST on this resource creates a new comment. The branch and language are in this case specified in the XML message.

4.2.3.6.1.8 /repository/document/<id>/comment/<id>

DELETE on this resource deletes a comment. The second <id> is the ID of the comment. To specify the document variant, add the optional request parameters branch and language.

Other methods are not supported on this resource.

4.2.3.6.1.9 /repository/document/<id>/availableVariants

A GET on this resource returns the list of variants that exist for this document.

4.2.3.6.2 Schema Management
4.2.3.6.2.1 /repository/schema/(part|field|document)Type

These resources represent the set of part, field and document types.

POST to these resources is used to create a new part, field or document type. The request body should contain an XML message conforming to the schemas found in fieldtype.xsd, parttype.xsd or documenttype.xsd.

4.2.3.6.2.1.1 Example scenario

This example illustrates how to create a new field type (with a selection list), simply by using the well-known "wget" tool.

This is the XML that we'll send to the server:

<?xml version="1.0"?>
<fieldType name="myNewField" valueType="string" deprecated="false"
           aclAllowed="false" size="0"
           xmlns="http://outerx.org/daisy/1.0">
  <labels>
    <label locale="">My New Field</label>
  </labels>
  <descriptions>
    <description locale="">This is a test field</description>
  </descriptions>
  <selectionList>
    <staticSelectionList>  
      <listItem>
        <labels/>
        <string>value 1</string>
      </listItem>
      <listItem>
        <labels/>
        <string>value 2</string>
      </listItem>
    </staticSelectionList>
  </selectionList>
</fieldType>

Let's say we save this in a file called newfieldtype.xml. We can then create the field by executing:

wget --post-file=newfieldtype.xml
     --http-user=testuser@1
     --http-passwd=testuser
     http://localhost:9263/repository/schema/fieldType

This supposes that "testuser" exists and has the Administrator role, which is required for creating field types.

Wget will save the response from the server in a file called "fieldType". The response is the same XML but now with some additional attributes such as the assigned ID. The response XML isn't pretty formatted, if you have libxml installed you can view it pretty using:

xmllint --format fieldType
4.2.3.6.2.2 /repository/schema/(part|field|document)Type/<id>

GET on these resources retrieves the XML representation of a part, field or document type.

POST on these resources updates a part, field or document type. The request body should then contain an altered variant of the XML retrieved via GET.

DELETE on these resources deletes them. Note that deleting types is only possible if they are not in use any more by any version of any document.

4.2.3.6.2.2.1 Example scenario

Let's take the previous field type example again, and add an additional value to the selection list. We first retrieve the XML for the field type (check the XML response of the previous sample to know the ID of the created field type): 

wget http://localhost:9263/repository/schema/fieldType/1

This will save a file called "1" (if that was the requested ID). To make it easier to work with, do a:

xmllint --format 1 > updatedfieldtype.xml

and change the updatedfieldtype.xml with an additional value in the selection list:

<?xml version="1.0" encoding="UTF-8"?>
<ns:fieldType xmlns:ns="http://outerx.org/daisy/1.0" size="0" updateCount="1"
  aclAllowed="false" deprecated="false" valueType="string"
  name="myNewField" lastModifier="3"
  lastModified="2004-09-09T09:06:51.032+02:00" id="1">
  <ns:labels>
    <ns:label locale="">My New Field</ns:label>
  </ns:labels>
  <ns:descriptions>
    <ns:description locale="">This is a test field</ns:description>
  </ns:descriptions>
  <ns:selectionList>
    <ns:staticSelectionList>
      <ns:listItem>
        <ns:labels/>
        <ns:string>value 1</ns:string>
      </ns:listItem>
      <ns:listItem>
        <ns:labels/>
        <ns:string>value 2</ns:string>
      </ns:listItem>
      <ns:listItem>
        <ns:labels/>
        <ns:string>value 3</ns:string>
      </ns:listItem>
    </ns:staticSelectionList>
  </ns:selectionList>
</ns:fieldType>

And then do:

wget --post-file=updatedfieldtype.xml
     --http-user=testuser@1
     --http-passwd=testuser
     http://localhost:9263/repository/schema/fieldType/1
4.2.3.6.2.3 /repository/schema/(part|field|document)TypeByName/<name>

GET on this resource retrieves a part, field or document type by its name.

You cannot POST on this resource, to update the type, use the previous (ID-based) resource.

4.2.3.6.3 Access Control Management
4.2.3.6.3.1 /repository/acl/<id>
4.2.3.6.3.2 /repository/filterDocumentTypes
4.2.3.6.4 Collection Management
4.2.3.6.4.1 /repository/collection/<id>
4.2.3.6.4.2 /repository/collectionByName/<name>
4.2.3.6.4.3 /repository/collection
4.2.3.6.5 User Management
4.2.3.6.5.1 /repository/user
4.2.3.6.5.2 /repository/role
4.2.3.6.5.3 /repository/user/<id>
4.2.3.6.5.4 /repository/role/<id>
4.2.3.6.5.5 /repository/userByLogin/<login>
4.2.3.6.5.6 /repository/roleByName/<name>
4.2.3.6.5.7 /repository/usersByEmail/<email>
4.2.3.6.5.8 /repository/userIds
4.2.3.6.6 Variant Management
4.2.3.6.6.1 /repository/branch
4.2.3.6.6.2 /repository/branch/<id>
4.2.3.6.6.3 /repository/branchByName/<name>
4.2.3.6.6.4 /repository/language
4.2.3.6.6.5 /repository/language/<id>
4.2.3.6.6.6 /repository/languageByName/<name>
4.2.3.6.7 Querying
4.2.3.6.7.1 /repository/query

GET on this resource is used to perform queries using the Daisy Query Language.

Required parameters:

Optional parameters:

4.2.3.6.7.2 /repository/facetedQuery

Used to perform a query for which the result contains the distinct values for the different items returned by the query. This allows to build a "faceted navigation" front end.

The query parameters are specified in an XML document which should be posted to this resource. The format of the XML is defined in facetedquery.xsd.

4.2.3.6.8 Other
4.2.3.6.8.1 /repository/userinfo

GET on this resource returns some information about the authenticated user. Takes no parameters.

4.2.3.6.8.2 /repository/comments

Usually comments are retrieved via the document they belong to, but it is also possible to get all comments of a user by doing a GET on this resource. Parameters:

4.2.3.7 Navigation Manager Extension

See also navigation.

/navigation

GET on this resource retrieves a navigation tree (customised for the authenticated user).

Parameters, all required unless indicated otherwise:

/navigationPreview

This resource allows to generate a navigation tree from a navigation source description specified as part of the request. This is used in the Daisy Wiki application to try out navigation trees before saving them.

Parameters:

/navigationLookup

Resolves a path against the navigation tree and returns the result of that lookup as an XML message. For more details, see the Java API (e.g. the class NavigationLookupResult).

Required parameters:

4.2.3.8 Publisher Extension

The purpose of the Publisher Extension component is to return in one call all the data needed to publish pages in the Daisy Wiki (or other front end applications).

/publisher/request

To this resource you can POST a publisher request. A publisher request takes the form of an XML document, and is described in detail over here.

/publisher/blob

GET on this resource retrieves a blob (the data of a part).

Required parameters:

The last modified, content type and content length headers are set.

4.2.3.9 Email Notifier Extension

The Email Notifier extension component makes available resources for managing the email subscriptions.

/emailnotifier/subscription/<id>

<id> is the ID of a user.

GET, POST, DELETE supported. XML Schema see subscription.xsd

/emailnotifier/subscription

GET on this resource returns all the subscriptions.

/emailnotifier/(document|schema|user|acl|collection)EventsSubscribers

GET on this resource returns all subscribers for the kind of event as specified in the request path. The returned information for each subscriber includes the user ID and the locale for the subscription.

In the case of documentEventSubscribers, the following additional request parameters are required:

The returned subscribers are then those that are explicitly subscribed for changes to that document, or those who are subscribed to a collection to which the document belongs, or those that are subscribed to all collections.

/emailnotifier/documentSubscription/<documentId>

This resource allows to manage document-based subscriptions without having to go through the full subscriptions.

Using POST on this resource allows to add or remove a subscription for the specified document for some user. Required parameters:

Using DELETE on this resource removes the subscriptions for the specified document for all users. This is useful to cleanup subscriptions when a document gets deleted. If the branch and language parameters are missing, the subscriptions for all variants fo the document will be removed, otherwise only for the specified variant.

/emailnotifier/documentSubscription/<documentId>/<userId>

This resource allows to quickly check if a user is subscribed for notifications to a certain document (using GET). Request parameters branch and language are required, specifying the ID of the branch and language.

/emailnotifier/collectionSubscription/<collectionId>

Only DELETE is supported on this resource, and deletes all subscriptions for the specified collection for all users.

4.2.3.10 Emailer Extension

The emailer extension allows to send emails. Only Administrators can do this.

/emailer

A POST to this resource will send an email, the following parameters are required:

4.2.3.11 Document Task Manager Extension

See also Document Task Manager.

/doctaskrunner/task

A GET on this resource retrieves all existing tasks for the current user, or the tasks of all users if the role is Administrator.

A POST on this resource is used to create a new task, in which case the body must contain an XML document describing the task (see also taskdescription.xsd).

/doctaskrunner/task/<id>

A GET on this resource retrieves information about a task.

A POST on this resource in combination with a request parameter "action" with value "interrupt" interrupts a task.

A DELETE on this resource deletes the persistent information about this task.

/doctaskrunner/task/<id>/docdetails

A GET on this resource retrieves detailed information about the execution of the task on the documents.

5 Daisy Wiki

5.1 General

5.1.1 Daisy Wiki Overview

An overview of the functionality of the Daisy Wiki can be found in the feature overview.

5.1.2 Daisy Wiki Sites

5.1.2.1 What is a Daisy Wiki "site"?

A Daisy Wiki site is a specific view on a Daisy Repository. A site is configured with a default collection (the concept of document collections is explained on the documents page). Full text searches and recent changes are automatically limited to only show documents from that default collection. New documents created via the site are by default assigned to that collection. Each site can have its own navigation tree, and is configured with a specific document as the homepage of the site.

A site is configured with a certain branch and language: for any document consulted via that site, the shown document variant will depend on that branch and language.

The Repository Server isn't aware of the concept of sites, nor does the site concept partition the repository in any way.

5.1.2.2 Defining sites

Sites are defined by creating a directory for the site and putting a siteconf.xml file in it. This directory should be created in the "sites" directory. By default, this sites directory is located at:

<wikidata directory>/sites

The location of this directory can be changed in the cocoon.xconf.

The content of the siteconf.xml file should strictly adhere to a certain schema, otherwise the site will be ignored (in that case, an error will be logged in Cocoon's log files). An example siteconf.xml is displayed below.

<siteconf xmlns="http://outerx.org/daisy/1.0#siteconf">
  <title>foobar</title>
  <description>The "foobar" site</description>
  <skin>default</skin>
  <navigationDocId>1</navigationDocId>
  <homepageDocId>2</homepageDocId>
  <!-- homepage>....</homepage -->
  <collectionId>1</collectionId>
  <contextualizedTree>false</contextualizedTree>
  <branch>main</branch>
  <language>default</language>
  <defaultDocumentType>SimpleDocument</defaultDocumentType>
  <publisherRequestSet>default</publisherRequestSet>
  <siteSwitching mode="all"/>
  <newVersionStateDefault>publish</newVersionStateDefault>
  <locking>
    <automatic lockType='pessimistic' defaultTime='15' autoExtend='true'/>
  </locking>
</siteconf>

Thus to create a new site, all you need to do is create a new subdirectory in the sites directory and put such a siteconf.xml in it.

Changes to the sites configurations are automatically picked up, it is not needed to restart the Daisy Wiki. It can take up to 10 seconds before Daisy notices your changes (this interval is configurable in the cocoon.xconf). If you don't see a site appearing, check the cocoon log files for errors.

The list of sites displayed to the user is filtered based on whether the user has access to the homepage document of the site. In case a custom homepage path is used (<homepage> instead of <homepageDocId>), you can still specify the homepageDocId to cause filtering. If this is not done, the site will always be displayed in the list.

5.1.2.3 Creating a new site using daisy-wiki-add-site

If you want to create a new site, including a new collection, a new navigation tree and a new homepage document, you can use the daisy-wiki-add-site program for this, which will automatically perform these steps for you and put a new siteconf.xml in the sites directory. To do this, open a command prompt, make sure DAISY_HOME is set, go to DAISY_HOME/install and execute:

daisy-wiki-add-site <location of wikidata directory>

5.1.2.4 Other site-features

5.1.2.4.1 skinconf.xml

Maintaining a custom skin can be more work then you'd like to put into it, therefore it is also possible to customise (or parameterise) existing skins. For example, for the default skin you can alter the logo in this way.

This is done by putting a file called skinconf.xml in the appropriate site directory. The contents of this file will be merged in the XML pipelines and hence be available to the XSL stylesheets. The required content and format of this file depends upon what the skin you use expects.

If a site-specific skinconf.xml is not provided, the system will use the skinconf.xml found in the root of the sites directory, if it exists.

For more information on skinconf.xml, see here.

5.1.2.4.2 Extension sitemaps

See Daisy Wiki Extensions.

5.1.3 Daisy Wiki Editor Usage Notes

5.1.3.1 Introduction

This document describes the editor used to modify pages stored in the document repository. The editor features wysiwyg editing.

5.1.3.1.1 Where do I find the editor?

The editor can be reached by either editing an existing document or creating a new document.

To edit an existing document, use the [Edit] link in the action menu of a document (by default, this is the menu shown on the right). This link is only visible if you are allowed to edit the document.

To create a new document, use the "New Document" link (in the top-right navigation). You are then first presented with a list of document types, after selecting one the editor will open.

5.1.3.1.2 Document Type Influence

The content of the edit screen depends somewhat on the document type of the document you're editing or creating. See documents for a general discussion on documents and document types. As a quick reminder, a document can consists of multiple parts and fields. The parts contain the actual content, the fields are for more structured (meta)data.

If a part is marked as a "Daisy HTML" part, you will be presented with a wysiwyg editor for that part. Otherwise, a file upload control will be shown. Because of the ability to plugin in custom part editors, other types of editors might also appear (e.g. for editing book definitions).

5.1.3.1.3 Supported browsers

In theory, the document editing screen should work on most browsers. However, to use wysiwyg editing, it is advisable to use a recent version of one of the mainstream browsers, Mozilla/Firefox or Internet Explorer. We do most of our testing using recent versions of Firefox and Internet Explorer 6.

On other browsers, the editor will fall back to a textarea allowing you to edit the HTML source. On browsers that support wysiwyg editing, you can also switch to source editing.

In any case, Javascript (and cookies) must be enabled.

5.1.3.1.4 Heartbeat

While editing a page, the server keeps some state about your editing session. After a certain period of inactivity, the server will clean up the editing session. To avoid the editing session to expire while you're working on a document, a 'heartbeat' signal keeps your session alive. (Technically speaking, this is a hidden frame that refreshes itself each 10 minutes).

5.1.3.1.5 Document locking

When you edit an existing document, the daisywiki will automatically take an exclusive lock on the document to ensure nobody else can edit the document while you're working on it. The duration of the initial lock is 15 minutes, the lock is then automatically extended if needed via the heartbeat signal.

If you start editing a page but decide you didn't want to after all, it is best to use the "Cancel editing" button at the bottom of the edit screen, so that your lock get cleared. If you don't do this, the lock will expire after at most 15 minutes, so this is not a big problem.

The locking behaviour can be adjusted by the site administrator. For example, the locking can be turned of completely. However, we expect that in most cases it will be left to the default behaviour described here.

5.1.3.1.6 Editing multiple documents at once

Editing multiple documents concurrently in different browser windows or tabs is supported.

5.1.3.2 Supported HTML subset and HTML cleaning

Although a wysiwyg editor is shown for the "Daisy HTML" parts, the goal is to limit the editing to a subset of HTML mainly focussing on structural aspects of HTML. So forget fonts, colors, special styling tricks, embedded javascript, and so on. Inserting those while editing in source view won't work either, as the HTML is cleaned up on the server side.

This cleanup process can also be triggered manually, by pressing the "Cleanup edited HTML" button. This can be useful if you pasted content copied from an external application and you want to see how it will look finally. When switching from wysiwyg to source view, the cleanup is also performed.

5.1.3.2.1 Supported HTML subset

These are the supported tags (or "elements") and attributes:

All tags not listed above will be removed (but their character content will remain). On the block-type elements and images, the id attribute is supported. For the most accurate list of elements and attributes, have a look at the htmlcleaner.xml file (see below).

The supported tags can have any content model as allowed by the HTML DTD, but of course limited to the supported tags. If an element occurs in a location where it is not supported, an ancestor is searched where it is allowed and the containing element(s) are ended, the element inserted, and the containing elements reopened. This happens for example when a <table> occurs inside a <p>.

<b> and <i> are translated to <strong> and <em> respectively, as are <span> tags with font-weight/font-style specifications.

If two or more <br> tags appear after one another, this is translated to a paragraph split. The meaningless <br>'s that the Mozilla editor tends to leave everywhere are removed. Text that appears directly in the <body> is wrapped inside <p> elements.

<br> tags inside <pre> are translated to newlines characters.

The result is serialized as a XML-well-formed HTML document (not XHTML) (UTF-8 encoded). Lines are split at 80 characters (if possible), meaningless whitespace is removed.

All this should also ensure that the resulting HTML is (mostly) the same whether it is edited using Mozilla or Internet Explorer.

The supported tags, attributes and classes for <p> are not hardcoded but can be configured in a file (htmlcleaner.xml). However, making arbitrary adjustments to this file is not supported (the html-cleaner code expects certain tags to be there). Adding new tags or attributes should generally not be a problem, but those won't have the necessary GUI editing support unless you implement that also.

5.1.3.3 Images

Images can be inserted either by browsing for an existing image in the repository, or by uploading a new image in the repository. You can also insert images that are not in the repository, but available at some URL.

You can change the alignment of the images (using the usual text-align buttons), and change how the text flows around the image. This last option won't have effect in the PDF output.

Note that images are also documents in the repository, thus are versioned and such. If you have an updated version of an image you want to insert, it is recommend to NOT delete the existing image and upload the new image, but rather go to the document editor for that image (you can use the "Open image document" button for this), and upload the new version over there.

Currently it is hardcoded that images use the document type called "Image" with the part called "ImageData".

5.1.3.4 Links

The format of links to other documents in the daisy repository is:

daisy:<document id>
for example:
daisy:167

The daisy link can furthermore include branch, language and version specifications:

daisy:<document id>@<branch name or id>:<language name or id>:<version id>

Each of these additional parts is optional. For example to link to version 5 of document 167, on the same branch and language as the current document variant, use:

daisy:167@::5

The <version id> can be a number or the strings last or live.

If you don't know the id of a document by heart (which is likely the case), use the "Create link by searching" button on the toolbar.

A link can furthermore contain a fragment identifier. A fragment identifier is used to directly link to a specific element (e.g. a heading or a table) in a document. For this you first need to assign an ID to the element you want to link to (there is an "Edit ID" for this on the toolbar), and then you can adjust the link. The link editor dialogs make it easy by allowing to browse for available element IDs.

5.1.3.5 Upload and link ("attachment")

TODO

5.1.3.6 Includes

It is possible to include other documents into the document you are editing. For this, choose "Include" in the style dropdown, and enter a "daisy:" link in the paragraph (there is a button on the toolbar to insert this link by searching). Behind the "daisy:" link, you can put some whitespace (e.g. a a space character), and then put whatever additional text you want. This text will not be published, but is useful to leave a comment (e.g. the name of the included document).

The system checks for recursive includes (when the document is published, not when it is edited). If a recursive include occurs, an error notice will be shown at the location of the include.

It is also possible to include other sources into your document, for example "http:" or "cocoon:" URLs (however, see Include Permissions). In that case, those URLs must produce an embeddable chunk of HTML in the form of well-formed XML. These includes are currently only supported in the HTML publishing, thus not for PDFs.

5.1.3.7 Embedded queries

It is possible to embed a query in a page. To do this, put your cursor on an empty line, and choose "Query" in the style dropdown. The style of the paragraph will change to indicate it will now be interpreted as a query. Then enter the query in the paragraph.

The query must be written in the Daisy Query Language. It is advisable to first try out your query via the "Query Search" page, and once it works and gives the expected results, to copy and paste it in the document.

If you save a document containing an invalid query, an error notice will be shown at the location of the query.

5.1.3.8 Query and Include

The "Query-Include" option allows to specify a query, and the documents returned by that query will be included (rather then showing the query results). This allows to quickly created an aggregated document without needing to manually insert includes.

It is not important what you put in the "select" part of the query, you can simply do "select id where ....".

5.1.3.9 IDs and fragment identifiers

It is not only possible to link to a document, but also to a specific location in a document. The element to which you want to link (a header, image, ...) must have an ID assigned. To do this, place the cursor inside the element to which to assign the ID, and then press the "Edit ID" button on the toolbar.

Then to link to the specific element, just insert a link like you always do. Both the "Create link" and "Create link by searching" dialogs allow to select the ID from the target document (in the "fragment ID" field). In the HTML source, the target ID is specified in the link as in this example:

daisy:5#notes

This link will cause the browser to scroll to the element with an ID attribute with the value "notes". The part starting from the hash sign is called the "fragment identifier".

Explicit anchor elements (e.g. HTML <a name="notes"/>) are not supported, as these sort of elements are not visible in the wysiwyg editor and thus users would work blindly if these were used (deleting or moving them without being aware of it, and being impossible to edit in wysiwyg mode).

5.1.3.10 Editor shortcuts

Shortcut

Function

ctrl+b

bold

ctrl+i

italic

ctrl+z

undo

ctrl+y

redo

ctrl+c

copy

ctrl+x

cut

ctrl+v

paste

ctrl+1, ctrl+2, ...

switch to header level 1, 2, ...

ctrl+a

select all

ctrl+q

switch between bullets / no bullets

ctrl+r

remove formatting (same as the gum icon) (since Daisy 1.1)

5.1.3.11 Editing hints

5.1.3.11.1 Firefox and Mozilla

Pressing enter once in Firefox inserts a newline (a <br>). To start a new paragraph, press enter twice.

The toolbar buttons for cut/copy/paste won't work because of security restrictions, though you can configure Firefox to allow this for a specific site. More information is given when you click on one of these buttons while in Firefox. However, using the keyboard shortcuts you can perform these operations without any special configuration.

When you add a link or apply a styling to some words on the end of a line, it might be difficult (read: impossible) to 'move after' the link or styling. You can interrupt the link or styling by moving the cursor to the end of the line, and pressing the 'Remove link' or 'Remove formatting' button (thus without making a selection).

5.1.3.11.2 Internet Explorer (IE)

Merging table cells in IE works a bit counter-intuitive. You cannot simply select multiple cells and click on the merge cell button. Instead, put the cursor in one cell, and click on the merge cell button. You will then be asked how many rows and columns you want to merge.

5.1.3.12 Character Set Information

By default, daisy is configured to use unicode (UTF-8) everywhere. For the part content you enter in the wysiwyg or source editor, you can use whatever unicode-supported characters (more correctly, it is limited by as far as Java supports unicode). Metadata however, such as the document name, fields, etc is stored in a relational database, MySQL, which needs to be configured with a certain encoding (in West-Europe often latin1) and hence is limited to the characters supported by that encoding. Contact your system administrator if you which to know what encoding that is, and thus to what characters (glyphs) you're limited.

5.1.4 Embedding multimedia

5.1.4.1 Introduction

It is possible to embed a flash animation, a movie or a sound fragment using the MultiMediaObject document type.

5.1.4.2 Usage

Create a new document, choose the document type MultiMediaObject, and upload the item. There are some fields available to control various options (like height, width and looping).

Then, in the document you want to embed the multimedia item, do an include of this document. This is done as follows:

5.1.4.3 Implementation

The MultiMediaObject document type is simply a regular document type with which a document type specific XSLT is associated which inserts the HTML <object> and <embed> tags.

5.1.5 Navigation

5.1.5.1 Overview

Daisy allows to create hierarchical navigation trees for your site. Some of the features and possibilities:

The Daisy Wiki has an advanced GUI for editing the navigation trees, so that users are not confronted with the raw XML. It is of course possible to switch to a source view. Editing a navigation tree is done in the same way as any other document is edited.

It is possible to create readable URLs (i.e. URLs containing readable names instead of numbers) by basing the URL space on the navigation tree and assigning meaningful node IDs to nodes in the navigation tree. See the document about URL management.

The 'root' navigation document of a site is accessible through the [Edit navigation] link below the navigation tree, which is visible is you are logged on as a non-guest-role user. You can also get an overview of all navigation documents using this query:

select id, branch, language, name where documentType = 'Navigation'

5.1.5.2 Description of the navigation XML format

The simplest possible navigation tree description is the empty one:

<d:navigationTree xmlns:d="http://outerx.org/daisy/1.0#navigationspec">
</d:navigationTree>

Adding document nodes to it is easy:

<d:navigationTree xmlns:d="http://outerx.org/daisy/1.0#navigationspec">
  <d:doc id="26"/>  
  <d:doc id="32">
    <d:doc id="15"/>  
  </d:doc>
</d:navigationTree>

As shown, the nodes can be nested.

By default , the navigation tree will display the name of the document as the label of a node. However, sometimes you might want to change that, for example if the name is too long. Also, when editing the navigation tree description as a source document, it will quickly become difficult to figure out what node stands for what. Therefore, you can add an attribute called "label" to the d:doc elements:

<d:navigationTree xmlns:d="http://outerx.org/daisy/1.0#navigationspec">
  <d:doc id="26" label="Introduction"/>  
  <d:doc id="32" label="Hot Stuff">
    <d:doc id="15" label="Fire"/>
  </d:doc>
</d:navigationTree>

By default the ID of a document node is the document ID, but you can assign a custom ID by specifying it in an attribute called nodeId. The custom ID should not start with a digit and not contain whitespace.

To link to a document on another branch or in another language, add a branch and/or language attribute on the d:doc element. The value of the attribute can be a branch/language name or ID. By default, documents are assumed to be on the same branch and in the same language as the navigation tree document itself.

To insert a link to an external location (a non-Daisy document), use the link element:

<d:navigationTree xmlns:d="http://outerx.org/daisy/1.0#navigationspec">
  <d:doc id="26" label="Introduction"/>  
  <d:doc id="32" label="Hot Stuff">
    <d:doc id="15" label="Fire"/>
  </d:doc>
  <d:link url="http://outerthought.org" label="Outerthought"/>
</d:navigationTree>

The attributes url and label are both required. The link element supports an optional id attribute.

If you want to group a number of items below a common title, use the group element. The group element can optionally have an attribute called id to specify a custom id for the node (otherwise, the id is automatically generated, something like g1, g2, etc).

<d:navigationTree xmlns:d="http://outerx.org/daisy/1.0#navigationspec">
  <d:group label="Some title">
    <d:doc id="26" label="Introduction"/>  
    <d:doc id="32" label="Hot Stuff">
      <d:doc id="15" label="Fire"/>
    </d:doc>
  </d:group>
  <d:link url="http://outerthought.org" label="Outerthought"/>
</d:navigationTree>

To import another navigation tree, use the import element:

<d:navigationTree xmlns:d="http://outerx.org/daisy/1.0#navigationspec">
  <d:group label="Some title">
    <d:doc id="26" label="Introduction"/>  
    <d:doc id="32" label="Hot Stuff">
      <d:doc id="15" label="Fire"/>
    </d:doc>
    <d:import docId="81"/>
  </d:group>
  <d:link url="http://outerthought.org" label="Outerthought"/>
</d:navigationTree>

The docId attribute on the d:import element is of course the id of the navigation document to be imported.

It is possible to dynamically insert nodes by including a query, for example: 

<d:navigationTree xmlns:d="http://outerx.org/daisy/1.0#navigationspec">
  <d:doc id="26" label="Introduction"/>  
  <d:doc id="32" label="Hot Stuff">
    <d:doc id="15" label="Fire"/>
    <d:query q="select name where $somefield='hot' order by name"/>
  </d:doc>
</d:navigationTree>

The selected value, in this example "name", will be used as the node label. If you select multiple values in the query, then group nodes will be created for the additional selected values, for example:

select documentType, $Category, name where true order by documentType, $Category, name

With this query, group nodes will be created per value of documentType, within that per value of $Category, and then finally document nodes with the name as label. Note that for meaningful results, you should add an order by clause which orders the values in the same sequence as the are selected. If you select fields, the fields shouldn't be multi-value fields.

Since the query is embedded in an XML file, don't forget that you might need to escape certain characters, e.g. < should be entered as &lt;

Queries embedded in a navigation tree are only executed once when the internal representation of the navigation tree is build. So if queries contain items that can change on each execution (such as the result of a CurrentDate() call), these will not work as expected.

The query element can have an optional attribute called filterVariants with value true or false. If true, the query results will be automatically limited to the branch and language of the navigation document.

If you want to automatically limit the result of queries in the navigation tree to documents contained by one ore more collections, you can add a collections element as first child of the navigationTree element:

<d:navigationTree xmlns:d="http://outerx.org/daisy/1.0#navigationspec">
  <d:collections>
    <d:collection name="MyCollection"/>
  </d:collections>
  <d:doc id="26" label="Introduction"/>  
  <d:doc id="32" label="Hot Stuff">
    <d:doc id="15" label="Fire"/>
    <d:query q="select name where $somefield='hot' order by name"/>
  </d:doc>
</d:navigationTree>

The doc, group, query, link and import nodes can be combined and nested as you desire, with the exception that query and import can't have child elements.

Any other elements besides the ones mentioned here are prohibited, as is text in between the nodes.

The doc, group and query nodes support a visibility attribute. This attribute can have one of three values: always (the default), hidden or when-active. Hidden nodes are never shown in the navigation tree, when-active nodes are only shown when active. Other than this, hidden and when-active nodes behave as usual, and can thus be used to influence the URL path or to let the navigation tree expand to a certain node.

5.1.5.3 Implementation notes

The Navigation Manager is implemented as an extension component running inside the repository server. It has its own HTTP+XML interface and remote Java API.

5.1.6 Faceted Browser

5.1.6.1 Introduction

The Daisy Wiki includes a faceted browser which allows for faceted navigation through the repository. The faceted browser shows the distinct values for selected properties (facets) of the documents in the repository, and allows to search for documents by selecting values for these facets. This technique is quite common in many websites, but Daisy's faceted browser makes it very easy to add it to your site.

A somewhat bland demo (i.e. only using system properties) of the faceted browser can be found on the main cocoondev.org site.

To use the faceted browser, you need to create a small configuration file in which you list the facets (document properties) to use. You can have multiple faceted navigation configurations.

5.1.6.2 Howto

Faceted navigations are defined on a per-site level. In the directory for the site, create a subdirectory called "facetednavdefs" if it does not already exist. Thus the location for this directory is:

<wikidata directory>/sites/<sitedir>/facetednavdefs

In this directory, create a file with the extension ".xml", for example "test.xml". The content of the file should be something like this:

<facetedNavigationDefinition xmlns="http://outerx.org/daisy/1.0#facetednavdef">
  <options>
    <limitToSiteCollection>false</limitToSiteCollection>
    <limitToSiteVariant>true</limitToSiteVariant>
    <defaultConditions>true</defaultConditions>
  </options>
  <facets>
    <facet expression="documentType"/>
    <facet expression="collections"/>
    <facet expression="lastModifierLogin"/>
  </facets>
</facetedNavigationDefinition>

About the content of this file:

The options limitToSiteCollection and limitToSiteVariant speak pretty much for themselves, they define whether the query should automatically limit to documents belonging to the collection, branch and language of the current site. If you want to include the collection or branch/language as facets to search on, then you put the respective options to false, otherwise to true. In this example, since we included collections in the list of facets, we put the limitToSiteCollection option to false.

The <facet> elements list the different facets on which the user can browse. The expression attribute contains an identifier as used in the Daisy Query Language. Thus to include document fields, use $fieldname.

The <defaultConditions> element is optional, and contain a set of a set of query conditions to limit the set of documents on which the faceted navigation will be done, for example:

<defaultConditions>$someField = 'abc' and $someOtherField='def'</defaultConditions>

The faceted navigation definition file is validated against an XML Schema, so don't put any additional elements in it or validation will fail.

Once you have saved this file, you can use the faceted browser immediately to browse on the defined facets (a restart of the Daisy Wiki is not needed). The faceted browser is accessed with an URL of this form:

http://localhost:8888/daisy/yoursite/facetedBrowser/test

In which you need to replace "yoursite" with the name of your site and "test" with the name of the file you just created, without the ".xml" extension.

5.1.7 URL space management in the Daisy Wiki

5.1.7.1 Overview

The URL space of the documents when published through a Daisy Wiki site is related to the hierarchical navigation tree of the site. With URL space we mean the URLs that get assigned to documents, or the other way around, what URL you need to enter in the location bar of your web browser to get a certain document. This document goes into some details about how all this works.

URL stands for Uniform Resource Locator. It is how resources (documents etc) on the world wide web are addressed, in other words the widely-known http://host.com/some/path things.

5.1.7.2 The (non-)relation between the Daisy repository and the URL space

The Daisy repository is a flat (non-hierarchical) document-store. It isn't necessarily web-related, and hence doesn't define, dictate or influence how documents are actually published on the web, including how these documents will map to the URL space.

Recall that documents in the repository are identified by a unique, numeric ID. The uniqueness is within one repository, it is a sequence number starting at 1, not a global unique identifier.

5.1.7.3 URL mapping

The URL mapping in the Daisy Wiki is based on the hierarchical navigation tree. This means that when a document is requested, the navigation tree is consulted to resolve the path. The other way around, when publishing documents, the logical daisy:<document-id> links that occur in documents that are stored in the repository are translated to the path at which they occur in the navigation tree (if a document occurs at multiple locations, the first occurrence -- in a depth-first traversal -- is used).

5.1.7.3.1 Relation between the navigation tree and the URL space

Each nested node in the navigation tree becomes a part of the path in an URL. The name of the part of the path is the ID of the navigation tree node. What this ID is depends on the type of node:

If you want to have more readable URLs, it is recommended to assign node IDs in the navigation tree. With readable URLs we mean URLs containing meaningful words instead of automatically assigned numbers.

5.1.7.3.2 Importance of readable URLs?

It is in no way required to assign custom node IDs in the navigation tree. You only need to do this if you want to have readable, meaningful URLs.

Some advantages of having readable URLs is:

However, URLs containing the raw document IDs also have their advantages:

It is a good idea to standardise on some conventions when naming navigation tree nodes. For example, use always lower case and separate names consisting of multiple parts with dashes.

If all you want to have are some shortcut URLs for certain documents, independent of where they occur in the navigation tree, you can run Apache in front of the Daisy Wiki and configure redirects over there.

5.1.7.3.3 How URL paths are resolved in the Daisy Wiki

When a request for a certain path comes in, the Daisy Wiki will ask the navigation tree manager to lookup that path in the navigation tree for the current site. There are a number of possible outcomes:

5.1.7.3.3.1 Site-search algorithm

The site search algorithm is used each time when a document might be more suited for display in the context of another site, thus when the document has not been found in the current site's navigation tree.

The sites that will be considered in the search can be configured in the siteconf.xml using the <siteSwitching> element, whose syntax is as follows:

<siteSwitching mode="stay|all|selected">
  <site>...</site>
  ... more <site> elements ...
</siteSwitching>

The mode attribute takes one of these values:

The site-search algorithm works as follows:

5.1.7.3.4 Not all documents must appear in the navigation tree

As a consequence of the above described resolving mechanism, any document can be accessed in the repository even if it does not occur in the navigation tree. Simply use an URL like:

http://host/daisy/mysite/<document-id>

In which <document-id> is the ID of the document you want to retrieve.

After each document URL you can add the extension .html, thus the above could also have been:

http://host/daisy/mysite/<document-id>.html

By default, the Daisy Wiki generates links with a .html extension, since this makes it easier to download a static copy of the site to the file system (otherwise you could have files and directories with the same name, which isn't possible).

5.1.8 Document publishing

5.1.8.1 Daisy Wiki Document Type Specific Styling

5.1.8.1.1 Introduction

Here we have a look at how to create custom XSLT stylesheets to render documents differently depending on their document type.

5.1.8.1.2 The Input XML

The input of the stylesheets is an XML document which has a structure as shown below. This is not an extensive schema containing every other element and attribute, but those that you'll need most often.

<document
    isIncluded="true|false"
    xmlns:d="http://outerx.org/daisy/1.0"
    xmlns:p="http://outerx.org/daisy/1.0#publisher">

  <context .../>
  <p:publisherResponse>
    <d:document xmlns:d="http://outerx.org/daisy/1.0"
        id="..."
        name="..."
        [... various other attributes ...] >

      <d:fields>
        <d:field typeId="..." name="..." label="..." valueFormatted="..."
                 [... other attributes and children ...]>
        ... more fields ...
      </d:fields>

      <d:parts>
        <d:part typeId="..." mimeType="..." size="..." label="..." daisyHtml="true/false">
          [... HTML content of the part including html/body if @daisyHtml=true ...]
        </d:part>
        ... more parts ...
      </d:parts>

      <d:links>
        <d:link title="..." target="..."/>
        ... more links ...
      </d:links>

      [... customFields, lockInfo, collectionIds ...]
    </d:document>
  </p:publisherResponse>
</document>

The root element, document, has an attribute isIncluded which indicates if this document is being published as top-level document or as include inside another document. Sometimes it can be useful to make a distinction between this.

The context element is the same as in the layout.xsl input. It provides access to various Wiki-context and user information. Before Daisy 1.5, the context element was not available, only a user element. The user element is still included for compatibility (not shown here) but will eventually be removed.

The p:publisherResponse element then contains the actual document (and possibly related information) to be published. It is the result of the p:preparedDocuments publisher request. It will always contain the basic d:document element but can contain additional information if a custom publisher request is used.

5.1.8.1.3 Expected stylesheet output

The output of the XSLT should be an embeddable chunk of HTML (or XSL-FO in the case of PDF). Thus no <html> and <body> elements, but something which can be inserted inside <body> (or inside a <div>, a <td >, etc). Where the produced output will end up depends on the stylesheet creating the general page layout, or in the case of included documents, the location of the inclusion.

5.1.8.1.4 Where the stylesheets should be put

The stylesheets should be placed in the following directory:

<wikidata directory>/resources/skins/<skin-name>/document-styling/<format>

In which <skin-name> is the name of the skin you're using (by default: "default"), and <format> either html or xslfo. Thus for the default skin, for HTML, this becomes:

<wikidata directory>/resources/skins/default/document-styling/html

The stylesheet should be named (case sensitive):

<document-type-name>.xsl
5.1.8.1.5 Example 1: styling fields in a custom way

Suppose we have a document type called "TestDocType" with a " SimpleDocumentContent" part , and two fields called "field1" and "field2". The default layout will first place the parts, then the fields (in a table), and then the out-of-line links (if any).

The stylesheet below shows how to put the fields at the top of the document: 

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:d="http://outerx.org/daisy/1.0">

  <xsl:import href="daisyskin:xslt/document-to-html.xsl"/>

  <xsl:template match="d:document">
    <h1 class="daisy-document-name"><xsl:value-of select="@name"/></h1>

    <p>
      Hi there! Here's the value of field1:
      <xsl:value-of select="d:fields/d:field[@name='field1']/@valueFormatted"/>
      and field 2:
      <xsl:value-of select="d:fields/d:field[@name='field2']/@valueFormatted"/>
    </p>

    <xsl:apply-templates select="d:parts/d:part"/>
    <xsl:apply-templates select="d:links"/>
    <!-- xsl:apply-templates select="d:fields"/ -->
  </xsl:template>

</xsl:stylesheet>

To minize our efforts, we import the default stylesheet and only redefine what is needed. For comparison, the default template for d:document looks as follows:

<xsl:template match="d:document">
  <h1 class="daisy-document-name"><xsl:value-of select="@name"/></h1>
  <xsl:apply-templates select="d:parts/d:part"/>
  <xsl:apply-templates select="d:links"/>
  <xsl:apply-templates select="d:fields"/>
</xsl:template>

This new stylesheet should be saved as:

<wikidata directory>/resources/skins/default/document-styling/html/TestDocType.xsl

Now surf to a document based on TestDocType, and you should see the result.

5.1.8.1.6 Example 2: styling parts in a custom way

In this example, suppose we have a document type called "Article" with parts "Abstract" and "Body". We would like to render the abstract in a box. The below stylesheet shows how this can be done.

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:d="http://outerx.org/daisy/1.0">

  <xsl:import href="daisyskin:xslt/document-to-html.xsl"/>

  <xsl:template match="d:document">
    <h1 class="daisy-document-name"><xsl:value-of select="@name"/></h1>

    <div style="margin: 20px; padding: 10px; border: 1px solid black; background-color: #ffd76c">
      <xsl:apply-templates select="d:parts/d:part[@name='Abstract']"/>
    </div>
    <xsl:apply-templates select="d:parts/d:part[@name='Body']"/>

    <xsl:apply-templates select="d:links"/>
    <xsl:apply-templates select="d:fields"/>
  </xsl:template>

</xsl:stylesheet>

5.1.8.2 Document information aggregation

5.1.8.2.1 Introduction

When a document is published in the Wiki, the Wiki will retrieve the document using a publisher request, and style it using a document-type specific stylesheet (or fall back to a default stylesheet).

The information available to this stylesheet is basically just the document with its parts and fields (and some wiki-context information such as the user etc.). Sometimes it might be useful to show additional information with the document.

For example, suppose you have a field "Category". When a document is published, you would like to show at the bottom of the document a list of documents which have the same value for the Category field as the document that is being published.

This can be done by making use of custom publisher request for these documents. The basic reference information on this can be found in the publisher documentation. Here we will have a look at how this applies to the Wiki using a practical example.

If you want to try out the example described here, then make a field Category (string), create a document type having this field (its name does not matter), and create a few documents of this document type, of which at least some share the same value for the Category field.

For none of the changes described here, it is required to restart the repository server or wiki.

5.1.8.2.2 Creating a publisher request set

The first thing to do is to define a new publisher request set. In the daisydata directory (not wikidata directory), you will find a subdirectory called "pubreqs":

<daisydata directory>/pubreqs

In this directory, create a new subdirectory, for the purpose of this example we will call it "foobar":

<daisydata directory>/pubreqs/foobar

In this directory, we need to create three files:

Let's start with the mapping file. Create, in the foobar directory, a file named mapping.xml, with the following content:

<?xml version="1.0"?>
<m:publisherMapping xmlns:m="http://outerx.org/daisy/1.0#publishermapping">
  <m:when test="$Category is not null" use="categorized.xml"/>
  <m:when test="true" use="default.xml"/>
</m:publisherMapping>

This mapping tells that when the document has a field Category, the publisher request in the file categorized.xml should be used. In all other cases, the second m:when will match and the publisher request in the file default.xml will be used. The expressions here are the same as used in the query language (and thus as in the ACL).

Instead of checking on the existence of a field, a more common case is to check on the document type. For this you would use an expression like documentType = 'MyDocType'.

Create (in the same foobar directory) a file called default.xml with the following content:

<?xml version="1.0"?>
<p:publisherRequest xmlns:p="http://outerx.org/daisy/1.0#publisher">
  <p:prepareDocument/>
  <p:aclInfo/>
  <p:subscriptionInfo/>
</p:publisherRequest>

This is the same publisher request as is normally used, when you do not bother to create custom publisher requests.

Now we arrive at the most interesting: the publisher request for documents having a Category field. Create a file called categorized.xml with the following content:

<?xml version="1.0"?>
<p:publisherRequest xmlns:p="http://outerx.org/daisy/1.0#publisher" styleHint="categorized.xsl">
  <p:prepareDocument/>
  <p:aclInfo/>
  <p:subscriptionInfo/>

  <p:group id="related">
    <p:performQuery>
      <p:query>select name where $Category = ContextDoc($Category) and id != ContextDoc(id)</p:query>
    </p:performQuery>
  </p:group>

</p:publisherRequest>

Note the difference with the publisher request in default.xml. We have now included a query which retrieves the documents with the same value for the Category field, but excluding the current document. The only purpose of the p:group element is to make it possible to distinguish this query from other queries we might add in the future.

Also note the styleHint attribute. This is an optional attribute that can be used to indicate the stylesheet to be used (instead of relying on the document-type specific styling).

5.1.8.2.3 Telling the Wiki to use the new publisher request set

In the Wiki, the publisher request set to be used can be specified per site (in the siteconf.xml) or can be changed for all sites (in the global siteconf.xml). Either way, this is done by adding a <publisherRequestSet> element in the siteconf.xml. For this example, we will change it globally, thus we edit:

<wikidata directory>/sites/siteconf.xml

And as child of the root element (the order between the elements does not matter), we add:

<publisherRequestSet>foobar</publisherRequestSet>

It can take a few seconds before the Wiki notices this change, but you do not need to restart the Wiki for this. If you would now go looking at documents with a Category field, they would still look the same as before, as we have not yet adjusted the stylesheets to display the new information.

5.1.8.2.4 Creating a stylesheet

The styling is just the same as with regular document-type specific styling, however the styling here is not specific to the document type but rather driven by the publisher request. In the publisher request we used the styleHint attribute to tell the Wiki it should use the categorized.xsl stylesheet. Other than that, everything is the same as for document-type specific styling. Thus we need to create a file categorized.xsl at the following location:

<wikidata directory>/resources/skins/default/document-styling/html/categorized.xsl

Since these stylesheets are in the same location as the document type specific stylesheets, care should be taken that their names do not conflict with document types (unless on purpose of course). If your custom publisher requests are related to the document type, it is not needed to specify the styleHint attribute as the normal document type specific styling will do its job.

Here is an example categorized.xsl:

<xsl:stylesheet version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:d="http://outerx.org/daisy/1.0"
  xmlns:p="http://outerx.org/daisy/1.0#publisher"> 
  
  <xsl:import href="daisyskin:xslt/document-to-html.xsl"/>

  <xsl:template match="d:document">
    <h1 class="daisy-document-name"><xsl:value-of select="@name"/></h1>
    <xsl:apply-templates select="d:parts/d:part"/>
    <xsl:apply-templates select="d:links"/>
    <xsl:apply-templates select="d:fields"/>

    <hr/>
    Other documents in this category:
    <ul>
      <xsl:for-each select="../p:group[@id='related']/d:searchResult/d:rows/d:row">
        <li>
          <a href="{$documentBasePath}{@documentId}?branch={@branchId}&amp;language={@languageId}">
            <xsl:value-of select="d:value[1]"/>
          </a>
        </li>
      </xsl:for-each>
    </ul>

    <xsl:call-template name="insertFootnotes">
      <xsl:with-param name="root" select="."/>
    </xsl:call-template>
  </xsl:template>
  
</xsl:stylesheet>

And that's it.

5.1.8.3 Link transformation

This section is about the transformation of links in the Daisy Wiki from "daisy:" to the public URL space.

5.1.8.3.1 Format of the links

It might be good to review the format of the links first. The structure of a Daisy link is:

daisy:docid@branch:language:version#fragmentid

The branch, language and version are all optional. Branch and language can be specified either by name or ID. The version is typically a version ID, or the string "live" (default) or "last" (to link to the last version). The fragment identifier is of course also optional.

A fragment identifier is used to point to a specific element in the target document.

If the branch and language are not mentioned, they are defaulted to be the branch and language of the document containing the link (and thus not to the default branch and language of the Daisy Wiki site).

A link can also consist of only a fragment identifier, to link to an element within the current document:

#fragmentid

The document addressed by a link is called the target document.

5.1.8.3.2 When and what links are transformed

The link transformation happens after the document styling XSLT has been applied.

The transformation applies to links in the following places:

5.1.8.3.3 Input for the document styling XSLT

Since the link transformation happens after the document styling, the document styling XSLT can influence the linking process (see "Linking directly to parts" below). For this, it is useful to know that the publisher leaves some information on links about the target document they link to. An annotated link looks like this:

<a href="daisy:123" p:navigationPath="/info/123">
  <p:linkInfo documentName="..." documentType="...">
    <p:linkPartInfo id="..." name="..." fileName="..."/>
  </p:linkInfo>
  see this
</a>

The whitespace and indenting is added here for readability, in reality no new whitespace is introduced (since the whitespace inside inline elements is significant).

The p:navigationPath attribute gives the path in the navigation tree where the document occurs. This attribute is only added if such a path exists, and if the navigation tree is known (i.e. if it is specified to the publisher, which in the Daisy Wiki is always the case). You should usually leave the p:navigationPath attribute alone, the link transformation process will use it to make the link directly point to the 'good' navigation location (afterwards, it will remove the p:navigationPath attribute).

The p:linkInfo element is only added if the target document exists and is accessible (i.e. the user has read permissions on it). It specifies the name of the target document and the name of its document type. Also, for each part in the document, a p:linkPartInfo element is added. Its id and name attribute specify the part type ID and part type name of the part. The fileName attribute is only added if the part has a file name.

It is the responsibility of the document styling XSLT to remove the p:linkInfo element. This is very simple with an empty template that matches on this element (as is the case in the default document-to-html.xsl).

5.1.8.3.4 Linking directly to parts

By default, the links will be transformed to links that point to the target document. This seems obvious, but sometimes it is desirable to link directly to the data of a part of the target document, for example for images. The link transformation process can be instructed to do so by leaving special attributes on the link element (<a> or <img>) in its namespace:

http://outerx.org/daisy/1.0#linktransformer

which is typically associated with the prefix lt.

The attributes are:

Examples of how to put this to use can be found in the document-to-html.xsl, more specifically look at how images and attachments are handled there.

5.1.8.3.5 Branch and language handling

If the branch and language differ from those specified in the site configuration, branch and language request parameters will be added.

5.1.8.3.6 Fragment ID handling

Fragment identifiers are prefixed with a string identifying the target document. This is needed because it is possible to publish multiple Daisy documents in one HTML page (e.g. using document includes), and the same element ID might be used in multiple documents, giving conflicts.

The format of the prefix is dsy<docid>_, an example prefixed fragment identifier looks like this:

#dsy123_hello

in which "123" is the ID of the target document, and "hello" the original fragment identifier target.

To make this work, the actual element IDs in Daisy documents are also prefixed with dsy<docid>_.

5.1.9 Daisy Wiki Skinning

Customising the look and feel of the Daisy Wiki is possible through:

Daisy ships with one skin called default.

The skin to use is configurable on the level of a site in the siteconf.xml file. Thus different sites can use different skins.

Pages not belonging to a particular site (such as the login screen, the sites index page, etc.) use a globally configured skin, defined in the global siteconf.xml file:

<wikidata directory>/sites/siteconf.xml

If this file does not exist, the skin called default will be used. The content of the global siteconf.xml file should be like this:

<siteconf xmlns="http://outerx.org/daisy/1.0#siteconf">
  <skin>default</skin>
</siteconf>

5.1.9.1 skinconf.xml

5.1.9.1.1 Introduction

The skinconf.xml file is simply an XML file which is merged in the general XML stream and is available to the XSLT stylesheets, most specifically the layout.xsl. This allows to pass configuration information to XSLT stylesheets. The actual supported configuration will be dependent on the XSLT, and thus on the skin. The supported configuration for the default skin is given below.

A skinconf.xml file can be put in the site directory, or in the global sites directory:

<wikidata directory>/sites/skinconf.xml

If a site doesn't have its own skinconf.xml file, it will fall back to using the global one.

5.1.9.1.2 default skin skinconf.xml
<skinconf>
  <logo>resources/local/mylogo.png</logo>
  <daisy-home-link>Daisy Home</daisy-home-link>
  <site-home-link>Site Home</site-home-link>
</skinconf>

Each of the parameters (= XML elements) is optional.

The parameters quite speak for themselves:

It can take up to 10 seconds before changes made to a skinconf.xml file are noticed.

5.1.9.2 Creating a skin

5.1.9.2.1 The anatomy of a skin

A skin consists of a set of files: CSS file(s), images, XSLT stylesheets, and possibly others which are grouped below one directory. The directory containing the skins is located at:

<wikidata directory>/resources/skins

The name of the skin is the name of the directory below the skins directory. On a blank Daisy install, this skins directory will be empty. The default skin can be found in:

<DAISY_HOME>/daisywiki/webapp/daisy/resources/skins/default
5.1.9.2.2 Creation of a dummy skin

Daisy has a fallback mechanism between skins, which means that a new skin can be created based on an existing skin. This makes the initial effort of creating a skin very small.

As an example, suppose you want to create a skin called coolskin. The minimal steps to do this are:

  1. Create a directory for the skin:
    <wikidata directory>/resources/skins/coolskin
  2. In this newly created directory, put a file called baseskin.txt containing just one line like this:
    default

    (this should be the very first line in that file)

    This specifies that the new skin will be based on the default skin. This means that any file which is not available in the coolskin skin, will instead be taken from the default skin. This allows a skin to contain only copies of those files that it wants to change.

    Although there is no directory called default in the skins directory, the system will transparently fall back to the skins directory in the Daisy Wiki webapp (mentioned above).

  3. Modify one or more siteconf.xml files to use the new skin. For the non-site specific pages (login screen, index page, ...) this is: 
    <wikidata directory>/sites/siteconf.xml

    Or for a specific site, the siteconf.xml file in the directory of that site.

Basically, you now have created a new skin, although it doesn't do anything yet. If you hit refresh in your browser, you will still see the same.

5.1.9.2.3 Customising the new skin

Now you can start customising the skin by copying files from the default skin and adjusting them. The two most important files, which allow to change most of the global look of the Daisy Wiki, are these:

  1. the <skindir>/css/layout.css file
  2. the <skindir>/xslt/layout.xsl file

If you only want to do smaller changes like changing some colours and fonts, you should get around by only copying the docstyle.css file to your new skin and adjusting it. (note: to change the logo, you can use the skinconf.xml mechanism)

The layout.xsl file builds the global layout of a page, thus how everything 'around' the main content shoud look. The input format of the XML that goes in the layout.xsl can be found here.

5.1.9.3 layout.xsl input XML specification

This is the layout.xsl input contract.

<page>
  <!-- The context element is usually produced by the PageContext class
       (but the layout.xsl doesn't care about this of course) -->
  <context>
    <!-- Information about the Daisy Wiki version -->
    <versionInfo version="..." buildHostName="..." buildDateTime="..."/>

    <!-- The mountPoint is everything of the URI path that comes before
         the part matched by the Daisy sitemap. By default, this is /daisy -->
    <mountPoint>...</mountPoint>

    <!-- The current 'version mode' -->
    <versionMode>live|last</versionMode>

    <!-- The site element specifies some information about the current Daisy Wiki site,
         this is of course only required when working in the context of a site. -->
    <site
      name="..."
      title="..."
      description="..."
      navigationDocId="..."
      collectionId="..."
      collection="..."
      branchId="..."
      branch="..."
      languageId="..."
      language="..."/>

    <!-- skinconf: the contents of the skinconf.xml file of the current site,
         or of the global skinconf.xml file in case the current page does is
         outside the context of a site. -->
    <skinconf/>

    <!-- user: information about the current user -->
    <user>
      <name>...</name>
      <login>...</login>
      <id>...</id>
      <activeRoles>
        <role id="..." name="..."/>
        (... multiple role elements ...)
      </activeRoles>
      <updateableByUser>true|false</updateableByUser>
      <availableRoles default="name of default role">
        <role id="..." name="..."/>
      </availableRoles>
    </user>

    <!-- layoutType: the type of layout that the layout.xsl must render.
         Tree possibilities:
            - default: the normal layout, possible with navigation tree,
                       page navigation links, links to other variants, etc.
            - mini: minimalistic layout, which shouldn't put the page content
                    inside a table (this layout is used by the editor screen,
                    and the HTMLArea in IE doesn't work when put inside a table.
            - plain: a layout that doesn't display anything beside the content.
    -->
    <layoutType>default|mini|plain</layoutType>

    <!-- request: some info about the request:
             - uri: the full request URI, including query string
             - method: GET, ...
             - server: scheme + host + port number if not 80
                       to which the HTTP request has been sent
    -->
    <request uri="..." method="..." server="..."/>

    <!-- skin: name of the current skin. Can be useful to use in paths to
         resources (images, css, js, ...) -->
    <skin>...</skin>
  </context>
  
  <!-- pageTitle: a title for the page, this is what comes inside the html/head/title
       element and thus in the title bar of the users' browser. This element may
       contain mixed content (e.g. i18n tags) so its content must be copied entirely,
       not just the string value. -->
  <pageTitle>...</pageTitle>

  <!-- layout hints (optional element):
        wideLayout: if there is no navigation tree and you want to make use of
                    the maximum available width, specify true. -->
  <layoutHints wideLayout="true"/>

  <!-- A hierarchical navigation tree as produced by the navigation manager. Optional. -->
  <n:navigationTree/>

  <!-- pageNavigation: these are page-specific links (actions). As for the pageTitle,
       the link/title element may contain mixed content.
       pageNavigation is optional. -->
  <pageNavigation>
    <link [needsPost="true"]>
      <title>...</title>
      <path>...</path>
    </link>
    (... more link elements ...)
  </pageNavigation>

  <!-- availableVariants: a list of other variants of the document displayed
       on the current page. Optional. -->
  <availableVariants>
    <variants>
      <variant href="..."
               branchName="..."
               languageName="..."
               [current="true"]/>
    </variants>
  </availableVariants>

  <!-- content: the actual content to be displayed on the page. The content
       of this element must be copied over literally into the resulting output. -->
  <content>
   ...
  </content>

  <!-- extraMainContent: additional content, which must be placed below the normal
       content, but only in the default layoutType. This is currently used to have
       comments displayed in the default layout but not in the plain layout. -->
  <extraMainContent>
    ...
  </extraMainContent>

  <!-- extraHeadContent: additional content which will be copied inside
       the <head> element -->
  <extraHeadContent>
    ...
  </extraHeadContent>

</page>

5.1.9.4 The daisyskin and wikidata sources

5.1.9.4.1 Introduction

The daisyskin and wikidata sources are Cocoon (Excalibur/Avalon) sources. Sources in Cocoon are additional schemes you can use in URLs. For example, for the daisyskin source this means you can use URLs of the form "daisyskin:something".

What makes the daisyskin and the wikidata sources special is that they have fallback behaviour. If a file is not found in a first location, it is searched for in another location. It is as if directories are transparently layered on top of each other.

The wikidata source only performs fallback between the wikidata directory and the Daisy Wiki webapp, while the daisyskin sources additionally can fall back to 'base' (parent) skins.

5.1.9.4.2 wikidata source

The wikidata source performs fallback between a wikidata directory and the Daisy Wiki webapp. Thus if a file is not found in the wikidata directory, it is taken from the Daisy Wiki webapp.

For example, the publication types for the books are accessed via the wikidata scheme. The Daisy Wiki has some built-in default publication types, and in addition you can define your own ones in the wikidata directory. By using the wikidata scheme, the Daisy Wiki sees both without requiring any special effort.

In some cases, it can be useful to refer directly to the file in the webapp. This can be done using a "(webapp)" hint in the URL:

wikidata:/(webapp)/path/to/file

This would basically be the same as not using the wikidata source at all. The advantage is that the 'wikidata' source knows about the location of the webapp and we keep working through the same abstraction, which at some point might prove useful.

One case where this might be useful is when you have an XSL in the wikidata dir that hides the original XSL in the webapp dir. In that case, you could include the hidden XSL in the hiding XSL like this:

<xsl:include href="wikidata:/(webapp)/path/to/my.xsl"/>

How the location of the actual wikidata directory  used by the wikidata scheme is determined, is described in Specifying the wikidata directory location.

5.1.9.4.3 daisyskin source

The daisyskin source:

The fallback first goes from wikidata to webapp, and then to the next base skin, as illustrated by the arrow in the image below.

5.1.9.4.3.1 URL structure

Most of the time one will use relative URLs like for example:

daisyskin:images/foo.png

This will load the file images/foo.png from the directory of the current skin. If this file isn't available in the skin, it will be loaded from this skins' base, or the base skin of that skin, and so on, until it is found (or not found).

Additionally, the URL can specify the name of the skin using the following syntax:

daisyskin:/(myskin)images/foo.png

This can be useful in you have an XSL in a custom skin, in which you want to import the corresponding XSL from the default skin.

And as last option, it is also possible to indicate that the file should always be taken from the webapp, without checking the wikidata directory:

daisyskin:/(myskin)(webapp)images/foo.png
5.1.9.4.3.2 Specifying the base skin

The base skin of a skin is specified by having a file called baseskin.txt in the directory of the skin. This file should simply contain one line (with no blanks before it) with the name of the base skin. The presence of the baseskin.txt file is optional, a skin isn't required to have a base skin. Recursive base skin references are of course not allowed (and are detected).

5.1.9.4.3.3 Note about caching

The content of the baseskin.txt files is cached, this cache is updated asynchronously, with a certain frequency, by default every 10 seconds. So if you change the content of a baseskin.txt file, it can take up to 10 seconds before the change will be noticed. The file-changed-checks are performed by a component called the ActiveMonitor, see the cocoon.xconf to change its execution interval.

Other than this, nothing is cached so file additions or removals are detected immediately.

5.1.10 Query Styling

5.1.10.1 Overview

By default, query results are rendered as a table. It is however possible to customize the styling of the query results. This is done by supplying a style_hint option in the query, for example:

select name where true option style_hint = 'bullets'

The style hint 'bullets' is included as a sample with Daisy, it styles the results as a bulleted list, taking the first selected value (here 'name') as the text to put next to the bullet.

5.1.10.2 Implementing query styles

Style hints are implemented using XSL. Implementing a style hint is a matter of adding a template to the query-styling-(html|xslfo).xsl files of a skin.

If you want the query styles to be available to all skins, it is recommended to add them to the query-styling-*.xsl of the default skin. Otherwise, add them to the skin of your choice.

The stylesheets for the query styling are located in the following directory: 

<wikidata directory>/resources/skins/<skin-name>/query-styling

Thus for the default skin, this is:

<wikidata directory>/resources/skins/default/query-styling

If this directory would not yet exist in your wikidata directory, you can simply create it.

Then in that directory, create a file called query-styling-html.xsl (if it doesn't exist already). The following shows an example of what the XSL could look like.

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:d="http://outerx.org/daisy/1.0">

  <xsl:include href="daisyskin:/(default)(webapp)query-styling/query-styling-html.xsl"/>

  <xsl:template match="d:searchResult[@styleHint='enum']">
    <ol>
      <xsl:for-each select="d:rows/d:row">
        <li>
          <a href="{$searchResultBasePath}{@documentId}.html?branch={@branchId}&amp;language={@languageId}">
            <xsl:value-of select="d:value[1]"/>
          </a>
        </li>
      </xsl:for-each>
    </ol>
  </xsl:template>

  <!-- Add other query-styling templates here -->

</xsl:stylesheet>

Here we implemented the query styling for the style hint "enum". It is basically the same as the bullets styling, but using an <ol> instead of <ul>.

Also note the special xsl:include. By creating this query-styling-html.xsl file for the default skin, we hide the one in the default skin in the webapp directory. However, we can import it in the current XSL using this instruction. The "(default)" in the href indicates the name of the skin (if not specified, the 'current skin' is used), and the "(webapp)" tells to explicitely use the file from the webapp directory, not the one from the wikidata directory (which would be the file we have created here, so if we didn't add the (webapp) we would have a recursive include).

Usually when an XSLT or one of the XSLTs included/imported by it changes, Cocoon should reload it. However, it seems this only works for the first level of includes, if included files themselve again include other XSLTs, it seems to stop working. Therefore changes to the file query-styling-html.xsl are not immediately reflected in the wiki. There is an easy work-around to avoid having to restart the wiki In order to let the changes take effect: simply touch
an XSLT which Cocoon will check for changes, in this case the searchresult.xsl (which is a first-level include in document-to-html.xsl). ('Touching' means updating the last modification time of the file, e.g. by saving it in an editor, or on unix using the touch command).

If you would add a query-styling-html.xsl to another skin than the default skin, and would like to include the stylings from the default skin, you can use the following include instruction in the XSL:

<xsl:include href="daisyskin:/(default)query-styling/query-styling-html.xsl"/>

Note that the href doesn't contain the "(webapp)" part, so that the daisyskin source will first check for a query-styling-html.xsl for the default skin in the wikidata directory.

For more information on the daisyskin source, see its documentation.

5.1.11 Daisy Wiki PDF Notes

5.1.11.1 Introduction

Daisy generates PDFs by feeding XSL-FO (a vocabulary for specifying formatting semantics) to Apache FOP (and XSL-FO processor). The PDFs are generated on the fly when a user requests them.

5.1.11.2 Images

FOP caches images, normally indefinitely. Daisy clears FOPs' image cache from time to time, by default every 5 minutes (this is configurable in the cocoon.xconf file). So if you upload a new version of an image, it can take up to 5 minutes before appearing in the PDF. Closing and reopening your browser would also give immediately the new variant, since images are retrieved on a per-session basis (the sessionid is encoded in the image URL).

5.1.11.3 Links

In PDF output, link URLs are mentioned in footnotes.

5.1.11.4 Layout limitations

Below we list the known limitations in PDF output.

5.1.11.4.1 Table column widths

FOP can't auto-balance column widths like web browsers do. Therefore, by default all columns in a table are made the same width, which might give undesirable results. However, you can assign custom column widths in the editor through the table settings dialog. Next to the absolute widths such as cm and mm, there's also the special unit * (star). This is for proportional widths. For example, assigning the columns of a two-column table the widths 1* and 2* respectively, will make the first column take one third of the width and the second two thirds.

Another possibilities is to assign a class to the table and use that to perform appropriate styling in a custom skin or document type specific XSLT.

5.1.11.4.2 Text flow around images

Text flow around images is not supported.

5.1.11.4.3 Table cell vertical alignment

Table cell vertical alignment is not supported.

5.1.12 Daisy Wiki Extensions

5.1.12.1 Introduction

The Daisy Wiki has some hooks to add your own functionality. You build extensions by building on top of Cocoon and making use of the available Daisy repository API (plus extension components such as the navigation manager and the publisher). To develop extensions, you don't need a Java development environment or knowledge, though you can if you want. The Daisy Wiki contains some samples to get you started.

Some examples of what you could do using extensions:

Daisy Wiki extensions are, just as the name implies, just extensions to the Daisy Wiki. If you want to develop a completely custom site, it is better to start from "scratch", albeit reusing some basic groundwork for interacting with the repository server and getting pages published. We are still working on providing a solution for this though, so in the meantime you can get along using the Daisy Wiki with skinning can extensions.

5.1.12.2 Basics

5.1.12.2.1 Where to put extensions

Extensions can be site-specific or shared by all sites.

Site-specific extensions are placed in a directory called cocoon in the directory of the site, thus:

<wikidata directory>/sites/<sitename>/cocoon

Cross-site extensions (shared by all sites) are placed in a directory called cocoon in the sites directory, thus:

<wikidata directory>/sites/cocoon
5.1.12.2.2 How extensions work

The Cocoon framework, on which the Daisy Wiki is build, has a concept called sitemaps. A sitemap describes how to handle a request, i.e. when a request is handled by Cocoon, Cocoon will consult the sitemap in order to know what it has to do. One sitemap can delegate to another sitemap, and that is how Daisy Wiki extensions are integrated in the main Daisy Wiki.

More specifically, all URLs within a site starting with ext/ are delegated to the extension sitemap: first the site-specific one and then the cross-site one.

Thus for the public URLs, this means all paths below:

http://localhost:8888/daisy/<sitename>/ext/

are handled by the extension sitemaps.

5.1.12.2.2.1 Some more details about the sitemap mounting

(If you are not familiar with Cocoon, don't be bothered that you don't understand all of this yet.)

The Daisy Wiki sitemap consists of two main blocks: one that handles internal requests and one that handles external requests. Each of these two blocks contains a matcher that will forward requests to the extension sitemap. This is illustrated in the following diagram.

In the diagram, the site-specific sitemap is also split into a part handling external requests and a part handling internal requests, but that is free choice of the implementer of the site-specific sitemap. An advantage of putting the internal requests pipeline above the one for external requests, is that for external requests the internal part will be skipped immediately, leading to a faster matching process. Another reason is that in the Daisy Wiki main sitemap, the pipeline for handling external requests starts with various initialisation actions, such as for determining the locale, looking up the site, ... By putting a separate pipeline for internal requests above the one for external requests, duplicate execution of that code is prevented in case of internal requests.

So in general, it is advisable that pipelines which will only be called internally (e.g. included in a document via the cocoon: protocol, or pipelines called from flow-logic), are put in the internal pipelines section.

5.1.12.3 Getting started

5.1.12.3.1 Creating your first extension

Here we show how to create a very simple and mostly useless "hello world" extension, just to illustrate some basics. We will create a Cocoon pipeline which generates a blurb of HTML showing "Hello world" and then show how to include that in a Daisy document.

5.1.12.3.1.1 Create a directory for your extension

We will create a cross-site extension. To make it easy to add multiple extensions in a modular way, we'll organize the extensions in subdirectories. Therefore, edit or create a sitemap.xmap file at the following location:

<wikidata directory>/sites/cocoon/sitemap.xmap

For a site-specific extension, follow the same instructions but use <wikidata directory>/sites/<sitename>/cocoon as the base location in which you do everything.

and put the following in this sitemap.xmap (if there is already something in it, just replace it with this):

<?xml version="1.0"?>
<map:sitemap xmlns:map="http://apache.org/cocoon/sitemap/1.0">

  <map:components>
  </map:components>

  <map:views>
  </map:views>

  <map:resources>
  </map:resources>

  <map:pipelines>

   <map:pipeline>
     <map:match pattern="*/**">
       <map:act type="ResourceExists" src="{1}/sitemap.xmap">
         <map:mount check-reload="yes" src="{../1}/sitemap.xmap" uri-prefix="{../1}"/>
       </map:act>
     </map:match>
   </map:pipeline>

 </map:pipelines>

</map:sitemap>

Then create a new subdirectory for your extension, lets say we call it mytest:

<wikidata directory>/sites/cocoon/mytest

With the sitemap.xmap file we just created, if you now request an extension URL starting with "mytest/", the cocoon/sitemap.xmap will try to forward the request processing to a sitemap located at cocoon/mytest/sitemap.xmap (this is what the map:mount does). We will create this sitemap in the next section.

5.1.12.3.1.2 Create a sitemap

In the above created directory (mytest), create a file called sitemap.xmap with the following content:

<?xml version="1.0"?>
<map:sitemap xmlns:map="http://apache.org/cocoon/sitemap/1.0">

  <map:components>
  </map:components>

  <map:views>
  </map:views>

  <map:resources>
  </map:resources>

  <map:pipelines>

   <map:pipeline type="noncaching">
     <map:parameter name="outputBufferSize" value="8192"/>

     <map:match pattern="hello">
       <map:generate src="hello.xml"/>
       <map:transform src="hello.xsl"/>
       <map:serialize type="xml"/>
     </map:match>

   </map:pipeline>

 </map:pipelines>
</map:sitemap>

The sitemap.xmap file is used by Cocoon to decide how to handle a request. This sitemap specifies that if the path matches "hello" (at least, the part of the path stripped from the part leading to this extension), then an XML-producing pipeline is executed which starts by reading and parsing the file hello.xml, transforming the parsed XML using the hello.xsl XSLT, and then finally serializing the result back to an XML document in the form of a byte stream (which is sent to the browser, or whoever made the HTTP request)

5.1.12.3.1.3 Create the file hello.xml

Still in the same directory, create a file called hello.xml with the following content:

<?xml version="1.0"?>
<hello>
  <helloText>Hello world!</helloText>
</hello>
5.1.12.3.1.4 Create the file hello.xsl

Also in the same directory, create a file called hello.xsl with the following content:

<?xml version="1.0"?>
<xsl:stylesheet
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  version="1.0">

  <xsl:template match="hello">
    <div style="border: 1px solid blue;">
      <xsl:value-of select="helloText"/>
    </div>
  </xsl:template>

</xsl:stylesheet>

For those unfamiliar with XSLT, this stylesheet defines a template which is executed when a hello element is encountered in the input. In that case, a div element is outputted with as content the value of the helloText element that is a child of the current hello element.

5.1.12.3.1.5 Try it out in your browser

Surf to the following URL (or equivalent), substituting <sitename> with the name of your Daisy Wiki site:

http://localhost:8888/daisy/<sitename>/ext/mytest/hello

If your browser shows XML (such as Firefox or IE), you will see this:

<div style="border: 1px solid blue;">Hello world!</div>
5.1.12.3.1.6 Include this in a Daisy document

To include this piece of HTML in a Daisy document:

5.1.12.3.2 Further pointers

You can now explore the various samples for further inspiration.

You'll see that many samples make use of Cocoon flowscript, which is basically Javascript with some Cocoon-specific APIs included (which also includes an advanced flow-control feature called continuations, but see the Cocoon documentation for that).

Daisy provides some additional functions for usage in flowscript, see the daisy-util.js reference.

In the sitemap, a line like this calls a function defined in a .js file:

<map:call function="minimalRss"/>

The function can then do some work (such as gathering data) and finally calls the sitemap again to show a page, using the sendPage function.

Many examples also make use of the Publisher, which is an extension component running inside the repository server. Its purpose and request format is described on its own page.

5.1.12.4 daisy-util.js API reference

To make the Daisy Wiki context and functionality easily available from flowscript (javascript), a small integration library called daisy-util.js is available.

5.1.12.4.1 Importing

To use it, add the following on top of the javascript file:

cocoon.load("resource://org/outerj/daisy/frontend/util/daisy-util.js");
5.1.12.4.2 The Daisy object

To avoid naming conflicts, all provided functions are wrapped in a "Daisy" object. To use any of the functions, create a Daisy object like this:

var daisy = new Daisy();

and then call the functions on the daisy object. The Daisy object is a lightweight object, so there's no need to store it in the session or pass it around to functions: just instantiate it if you need it.

5.1.12.4.3 Functions
5.1.12.4.3.1 daisy.getRepository()

Returns the Repository object for the current user. Using the repository object, you can perform any repository operation, from creating documents to performing queries. See the javadoc API included in the binary Daisy distribution.

This is the method you should most often use, the methods below are for special cases.

5.1.12.4.3.2 daisy.getRepository(login, password)

Returns the Repository object for an arbitrary user.

5.1.12.4.3.3 daisy.getGuestRepository()

Returns the Repository object for the guest user. This can be useful if you want to do something specifically as guest.

If the current user is "not logged in" (which means, is automatically logged in as guest user), you can simply use the getRepository() method.

5.1.12.4.3.4 daisy.getPageContext(repository)

The repository argument is optional, by default the repository object of the current user will be used.

The PageContext object is an object encapsulating various context information and is streamable as XML. An instance of PageContext is usually passed on to the view layer and streamed by a JX template (see the ${pageContext} in the JX templates). This automatically generates the context element that is required as input to the layout.xsl.

5.1.12.4.3.5 daisy.resolve(uri)

Resolves an URL to a more absolute form. There is nothing Daisy-specific about this method, but this can be useful when reusing a pipeline from the main Daisy Wiki sitemap while specifying a file located in the current extension. This is used in the guestbook sample.

5.1.12.4.3.6 daisy.getHTMLCleaner(configFilePath)

Returns a HTMLCleaner object. The configFilePath is optional, and specifies the path to a htmlcleaner.xml file. By default, the default htmlcleaner.xml of the Daisy Wiki will be used.

The HTMLCleaner object allows to clean up and normalize HTML in the same way as is otherwise done when editing through the default document editing screen. The most common usage is:

var daisy = new Daisy();
var content = "<html><body>Hello world!</body></html>";
var htmlcleaner = daisy.getHTMLCleaner();
var data = htmlcleaner.cleanToByteArray(content);

The data object, which is a Java byte array, can then be supplied to the setPart() method of the document object (see the repository API javadoc).

5.1.12.4.3.7 daisy.performPublisherRequest(pipe, params, publishType, repository)

Performs a publisher request. The publisher request is build by executing the Cocoon pipeline specified by the pipe argument. The params argument is supplied as "viewData" to this pipeline. Such a pipeline will typically use the JX template generator.

The publisher request built from the pipeline is then executed by the publisher.

If the request contained any <p:preparedDocuments> elements having an attribute applyDocumentTypeStyling="true", then the prepared documents included in this element will be extracted, document type specific styling will be applied to them, and the result will be put aside in a request attribute for later merging by use of the DaisyIncludePreparedDocuments transformer.

The result of executing the publisher is returned as a SAXBuffer object, this is an object which can easily be streamed in a pipeline using the JX template generator (or a file generator with src="xmodule:flow-attr:something").

The publishType argument identifies the kind of document type specific styling that should be applied, this is either html or xslfo. This argument is only relevant if the publisher request contains any <p:preparedDocuments applyDocumentTypeStyling="true"/> requests.

The repository argument is optional, by default the repository object of the current user is used.

5.1.12.4.3.8 daisy.buildPublisherRequest(pipe, params)

This builds a publisher request, the same way as is done by daisy.performPublisherRequest, but doesn't execute it. This can be useful for debugging purposes: call buildPublisherRequest with the same parameters as you would call performPublisherRequest, and dump it to, for example, the console: 

var publisherRequest = daisy.buildPublisherRequest(pipe, params);
java.lang.System.out.println(publisherRequest);

Of course, if you do this for debugging purposes, be sure to remove these calls afterwards!

BTW, for the curious, to execute the publisher request yourself, the scenario is like this:

var daisy = new Daisy();
var publisherRequest = daisy.buildPublisherRequest(....);
var repository = daisy.getRepository();
var publisher = repository.getExtension("Publisher");
publisher.processRequest(publisherRequest, ..some sax contenthandler..);

If you use the performPublisherRequest method, the "..some sax contenthandler.." will be a ContentHandler that automatically does the document type specific styling stuff.

5.1.12.4.3.9 daisy.getSiteConf()

Returns an object of the following type:

org.outerj.daisy.frontend.components.siteconf.SiteConf

for the current site. This allows access to various site parameters such as its default collection.

5.1.12.4.3.10 daisy.getMountPoint()

Returns the part of the URL path leading to the main Daisy sitemap (thus, up until where the URLs are interpreted by the Daisy Wiki). On a default deployment, this is /daisy.

5.1.12.4.3.11 daisy.getDaisyContextPath()

Returns the URL of directory containing the main Daisy sitemap, thus something like file:/..../daisywiki/webapp/daisy/. This can be useful to access Daisy Wiki resources from extensions.

5.1.12.4.3.12 daisy.getDaisyCocoonPath()

Returns the part of the URL path that is interpreted by Cocoon and leads to the Daisy Wiki sitemap. On a default deployment, this is /daisy. This is useful to call pipelines in the main Daisy Wiki sitemap.

To illustrate the difference with getMountPoint: if the servlet context path would for example be /cocoon, then:

daisy.getMountPoint() would return /cocoon/daisy

and

daisy.getDaisyCocoonPath() would return /daisy

5.1.12.4.3.13 daisy.getLocale()

Returns the java.util.Locale object for the current user's locale.

5.1.12.4.3.14 daisy.getLocaleAsString()

Returns the locale as string.

5.1.12.4.3.15 daisy.getVersionMode()

Returns the active "version mode", which is either 'live' or 'last'. This indicates which version of documents the user wants to see by default.

The returned object is a WikiVersionMode object, converting it to string gives either 'live' or 'last'.

5.1.12.5 Publisher

5.1.12.5.1 Introduction

The publisher is an optional component that runs inside the repository server. Its goal is to retrieve in one remote call the information you want to display on a page. The result is returned as an XML document. The requestable information consists of more than just XML dumps of repository entities, the publisher provides all sorts of extra functionality, such as 'prepared documents' for publishing and diffs between document versions.

A publisher-request takes the form of an XML document which describes the various stuff you want the publisher to return. The remainder of this document describes the format of these XML requests.

The publisher was developed to support the needs of the Daisy Wiki, but can be useful for other applications as well.

5.1.12.5.2 The publisher request format

This section describes the format of the publisher requests. The responses are not described in much detail, they are easily viewable by trying the requests out on a live Daisy instance. This can, for example, be done by creating a file containing a publisher request and using a tool like wget to send it to the HTTP interface of the publisher, thus like this:

wget --post-file=pubreq.xml --http-user=testuser@1
     --http-passwd=testuser http://localhost:9263/publisher/request
5.1.12.5.2.1 p:publisherRequest

A basic, empty publisher request is structured as follows:

<p:publisherRequest
    xmlns:p="http://outerx.org/daisy/1.0#publisher"
    locale="en-US"
    versionMode="live"
    exceptions="throw">

 [... various publisher requests ...]

</p:publisherRequest>

The reponse of a publisher request is structured as follows:

<p:publisherResponse
    xmlns:p="http://outerx.org/daisy/1.0#publisher">

 [... responses to the various requests ...]

</p:publisherResponse>

About the attributes on the p:publisherRequest element:

The locale attribute is optional, by default the en-US locale will be used. If the publisher request is executed as part of another publisher request (see p:preparedDocuments), the locale will default to the locale of the 'parent' publisher request.

The versionMode attribute is optional, valid values are live (the default) or last. This attribute indicates which version of a document should be used by default if no explicit version is indicated. If its value is 'last', it will also cause the option 'search_last_version' to be set for various queries (those embedded in documents when requesting a preparedDocument, those executed by p:performQuery or p:forEach). It will also influence the version mode of the navigation tree when using p:navigationTree.

The exceptions attribute is also optional, throw is its default value. Basically this means that if an exception occurs during the processing of the publisher request, it will be thrown. It is also possible to specify inline, in which case the error description will be embedded in the p:publisherResponse element, but no exception will be thrown.

The styleHint attribute (optional, not shown above) will simply be replicated on the p:publisherResponse element. The publisher itself does not interpret the value of this attribute. See other information on document styling for when this is useful.

5.1.12.5.2.2 p:document

A p:document request is used to change the context document. The context document is the document on which the document related requests listed in the next section apply.

Any publisher request element can be nested within p:document.

The p:document request can work in three ways

5.1.12.5.2.2.1 Specify the document explicitly

Using the attributes id, branch (optional), language (optional) and version (optional) the new context document is specified.

The branch and language can be specified either by name or ID. If not specified, they default to the main branch and default language.

The version can be specified as "live" (the default), "last" or an explicit version number.

5.1.12.5.2.2.2 Specify a field attribute

When this p:document is used in a location where there is already a context document (e.g. from a parent p:document), it is possible to use an attribute called field. The value of this attribute should be the name of a link-type field. This p:document request will then change the context document to the document specified in that link-type field in the current context document. If the current context document does not have such a field, the p:document request will be silently skipped. If the link-type field is a multivalue field, the p:document request will be executed once for each value of the multivalue field. This will then lead to multiple sibling p:document elements in the publisher response.

5.1.12.5.2.2.3 Implicit

When the p:document request is used as child of the p:forEach request (see later), the context document is determined by the current document of the for-each loop.

5.1.12.5.2.2.4 Output of p:document

The p:document request will output a p:document element in the publisherResponse. This element will have attributes describing the exact document variant and version that the context document was changed to.

5.1.12.5.2.3 Requests that can only be used when a context document is available
5.1.12.5.2.3.1 p:aclInfo

Evaluates the document against the ACL and returns the result. This request requires no attributes.

5.1.12.5.2.3.2 p:subscriptionInfo

Returns whether the user is subscribed for email notifications for this document, the result is a tag like this:

<p:subscriptionInfo subscribed="true|false"/>

This request requires no attributes.

5.1.12.5.2.3.3 p:comments

Returns the comments for the current document. Newlines in the comments are replaced with <br/> tags. This request requires no attributes.

5.1.12.5.2.3.4 p:availableVariants

Returns the available variants for this document. This request requires no attributes.

5.1.12.5.2.3.5 p:preparedDocuments (& p:prepareDocument)

p:preparedDocuments is the most important of all publisher requests. It returns the content of the document prepared for publishing. The preparation consists of all sorts of things such as:

The full format of the request is:

<p:preparedDocuments applyDocumentTypeStyling="true|false" publisherRequestSet="...">
  <p:navigationDocument id="..." branch="..." language="..."/>
</p:preparedDocuments>

The applyDocumentTypeStyling attribute isn't used by the publisher, but is simply replicated in the result. In the Daisy Wiki, its purpose is to indicate whether document type specific styling should be automatically applied.

The publisherRequestSet attribute: see below.

The p:navigationDocument element is optional. If supplied, it enables to annotate "daisy:" links with the target path in the navigation tree.

5.1.12.5.2.3.5.1 How it works

p:preparedDocuments looks up a new publisher request to be performed on the context document. The publisher request to be used can be determined dynamically (described further on), but by default it is this one:

<p:publisherRequest xmlns:p="http://outerx.org/daisy/1.0#publisher">
  <p:prepareDocument/>
  <p:aclInfo/>
  <p:subscriptionInfo/>
</p:publisherRequest>

This publisher request should (usually) contain a <p:prepareDocument/> element. The p:prepareDocument will be replaced by the Daisy document as XML (d:document), in which the HTML content is inlined and processed (i.e. the things mentioned in the enumeration above). If the content contains an include of another document, then for this included document the publisher will again determine a publisher request to be performed upon it, and execute it. The same happens for each include (recursively). The results of all these publisher requests are inserted in the publisher response in a structure like this:

<p:preparedDocuments applyDocumentTypeSpecificStyling="true|false">

  <p:preparedDocument id="1">
    <p:publisherResponse>
      <d:document ...
    </p:publisherResponse>
  </p:preparedDocument>

  <p:preparedDocument id="2">
    <p:publisherResponse>
      <d:document ...
    </p:publisherResponse>
  </p:preparedDocument>

</p:preparedDocuments>

The publisher response of the context document will always end up in the p:preparedDocument element with attribute id="1".  If the document includes no other documents, this will be the only p:preparedDocument. Otherwise, for each included document (directly or indirectly), an additional p:preparedDocument element will be present.

So the included documents are not returned in a nested structure, but as a flat list. This allows to perform custom styling on each separate document before nesting them.

On the actual location of an include, a p:daisyPreparedInclude element is inserted, with an id attribute referencing the related p:preparedDocument element.

The content of a p:preparedDocument element is thus a single p:publisherResponse element, which in turns contains a single d:document element (as result of the p:prepareDocument in the publisher request). This d:document element follows the standard form of XML as is otherwise retrieved via the HTTP interface or by using the getXml() method on a document object, but with lots of additions such as inlined content for Daisy-HTML parts and non-string attribute values formatted according to the specified locale.

If you requested the live version of the document, but the document does not have a live version, there will simply be no p:preparedDocuments element in the response.

5.1.12.5.2.3.5.2 Determination of the publisher request to be performed

If instead of the default publisher request mentioned above, you want to execute some custom publisher request (which can be used to retrieve information related to the document being published), then this is possible by defining a publisher request set.

In the data directory of the repository server, you will find a subdirectory called 'pubreqs'. In this directory, each subdirectory specifies a publisher request set. Each such subdirectory should contain a file called mapping.xml and one or more other files containing publisher requests.

The mapping.xml file looks like this:

<m:publisherMapping xmlns:m="http://outerx.org/daisy/1.0#publishermapping">
  <m:when test="documentType = 'mydoctype'" use="myrequest.xml"/>
  <m:when test="true" use="default.xml"/>
</m:publisherMapping>

The publisher will run over each of the when rules, and if the expression in the test attribute matches, it will use the publisher request specified in the use attribute. The expressions are the same as used in the query language, and thus also the same as used in the ACL definition.

To make use of such a specific set of publisher requests, you use the publisherRequestSet attribute on the p:preparedDocuments element. The value of this attribute should correspond to the name of subdirectory of the pubreqs directory.

In the Daisy Wiki, the publisher request set to be used can be specified in the siteconf.xml

5.1.12.5.2.3.5.3 p:prepareDocument

The p:prepareDocument can have an optional attribute called inlineParts. This attribute specifies a comma-separated list of part type names (or IDs) for which the content should be inlined. By default this only happens for parts for which the Daisy-HTML flag is set to true.

The inlining will only happen if the actual part has a mime-type that starts with "text/" or if the mime-type is recognized as an XML mime-type. Recognized XML mime-types are currently text/xml, application/xml or any mime type ending with "+xml".

If it is an XML mime-type, then the content will be parsed and inserted as XML. Otherwise, it will be inserted as character data (assuming UTF-8 encoding of the part text data). If the inlining actually happened, an attribute inlined="true" is added to the d:part element in question.

5.1.12.5.2.3.6 p:shallowAnnotatedVersion

Returns the shallow version XML (this the version XML without field and part information in it), thus a "d:version" element. This request requires no attributes.

If you requested the live version of the document, but the document does not have a live version, there will simply be no d:version element in the response.

5.1.12.5.2.3.7 p:annotatedDocument

Returns the document XML with some slight annotations, thus a d:document element. This request requires no attributes.

In contrast with most other elements, if you request the live version of the document but the document doesn't have a live version, this element will automatically fallback to the last version, so there will always be a d:document element in the response.

5.1.12.5.2.3.8 p:annotatedVersionList

Returns a list of all versions of the document (as a d:versions element). This request requires no attributes.

5.1.12.5.2.3.9 p:diff

Returns a diff of the document version with another version of this document or another document.

The full form of this request is:

<p:diff>
  <p:otherDocument id="..."  branch="..." language="..." version="..."/>
</p:diff>

If no p:otherDocument element is specified, the diff will automatically be taken with the previous version of the document. If there is no such version (because the document has only one version yet), there will be no diff response.

If a p:otherDocument element is supplied, any combination of attributes is allowed, in other words all attributes are optional.

5.1.12.5.2.4 p:navigationTree

Returns a navigation tree. The full form of this request is:

<p:navigationTree>
  <p:navigationDocument id="..." branch="..." language="..."/>
  <p:activeDocument id="..." branch="..." language="..."/>
  <p:activePath>...</p:activePath>
  <p:contextualized>true|false</p:contexutalized>
  <p:versionMode>live|last</p:versionMode>
</p:navigationTree>

The activeDocument, activePath and versionMode elements are optional.

5.1.12.5.2.5 p:myComments

Returns a list of all private comments of the user.

5.1.12.5.2.6 p:performQuery

Returns the result of executing a query. The full form of this request is: 

<p:performQuery>
  <p:query>...</p:query>
  <p:extraConditions>...</p:extraConditions>
</p:performQuery>

The p:extraConditions element is optional, and specifies additional conditions that will be AND-ed with those in the where clause of the query. This feature is not often needed. It can be useful when you let the user enter a free query but want to enforce some condition, e.g. limit the documents to a certain collection.

The result of a query is a d:searchResult element.

If there is a context document available (i.e. if this p:performQuery is used inside a p:document) then the ContextDoc function can be used in the query.

5.1.12.5.2.7 p:forEach

The p:forEach element allows to perform a p:document request for each document (actually, document variant) returned by query. The full form of this request is:

<p:forEach useLastVersion="true|false">
  <p:query>...</p:query>
  <p:document .... />
</p:forEach>

If the useLastVersion attribute is false or not specified, the live version of each document will be used, otherwise the last version.

If there is a context document available (i.e. if this p:performQuery is used inside a p:document) then the ContextDoc function can be used in the query.

5.1.12.5.2.8 p:if

Allows to execute a part of the publisher request only if a certain test is satisfied. Syntax:

<p:if test="...">
  [... any publisher request ...]
</p:if>

The test attribute specifies a conditional expression (an expression evaluating to true or false) in the same format as used in the Daisy query language.

5.1.12.5.2.9 p:choose
<p:choose>
  <p:when test="...">
     [... any publisher request ...]
  </p:when>

  [... more p:when's ...]

  <p:otherwise>
    [... any publisher request ...]
  </p:otherwise>
</p:choose>

There should be at least one p:when, the p:otherwise is optional.

5.1.12.5.2.10 p:group

The p:group element performs no function by itself, except to act as a container for other requests. It needs an id attribute:

<p:group id="...">

 [... other requests here ...]

</p:group>

It allows to distinguish between e.g. different queries or navigation tree results if you would have more then one of them.

5.1.12.5.2.11 p:resolveDocumentIds

This element allows to retrieve the names of a set of documents of which you have only the ID. The advantage compared to using simply the repository API is that this only requires one remote call for as many documents as you need (assuming you are using the remote API, otherwise it does not make a difference).

<p:resolveDocumentIds branch="..." language="...">
  <p:document id="..." branch="..." language="..." version="..."/>
</p:resolveDocumentIds>

The branch and language attributes on the p:resolveDocumentIds element specify the default branch and language to use if it is not specified on the individual documents. These attributes are optional, the main branch and default language is used as default when these attributes are not specified.

The branch, language and version attributes on the p:document element are optional. By default the live version is used, or the last version if the document does not have a live version.

The result has the following format:

<p:resolvedDocumentIds>
  <p:document id="..." branch="..." language="..." version="..." name="..."/>
</p:resolvedDocumentIds>

The id, branch, language and version attributes are simply copied from the requesting document element. The name attribute is added. The p:document elements in the result corresponds to those in the request at the same position.

If there is some error (such as the document does not exist, or the specified branch or language does not exist), an error message is put in the name attribute.

5.1.12.6 Document editor initialisation

5.1.12.6.1 Introduction

Usually users create new documents by choosing a document type on the "New document" page. Sometimes it can be useful to create documents from other places, or to have the editor initialised with certain content (e.g. a field with a value already assigned or so). Here we have a look at the various ways to launch the document editor for creation of a new document.

5.1.12.6.2 The basics

To open the document editor for the creation of a new document, do a HTTP POST operation to the following URL:

http://localhost:8888/daisy/<sitename>/new/edit

Of course, change the host name and port number, the mount point ("/daisy") and the sitename to match your situation. This URL needs to be supplied with different parameters depending on what you want to, as described in the next sections.

5.1.12.6.2.1 startWithGet parameter

In some cases, you may want to redirect to the document editor from some server-side code. In such cases it is not possible to do a HTTP POST operation. Therefore, a special startWithGet parameter is available, to be used as follows:

http://localhost:8888/daisy/<sitename>/new/edit?startWithGet=true
5.1.12.6.2.2 returnTo parameter

Normally, when the user is done editing, the user will be shown the document that was just edited (or, when pressing cancel and it was a new document, the site's home page will be shown). It is possible to control where the user will be brought to after editing a document by means of a returnTo request parameter:

http://localhost:8888/daisy/<sitename>/new/edit?returnTo=/some/path/of/your/choice

This is useful when the edited document is part of some aggregated display, and you want to bring the user back to the aggregated page, rather then edited document.

5.1.12.6.3 Create a new document of a certain type

This is the most common case. The editor will be opened with a blank document of a certain type. The document type needs to be specified in a parameter called documentType, either by name or ID.

Thus for example, doing a POST to this URL will open a new editor for a SimpleDocument-type document

http://localhost:8888/daisy/<sitename>/new/edit?documentType=SimpleDocument

Optionally, branch and language parameters can be added if they would differ from the site's default.

5.1.12.6.4 Create a new document starting from a template document

This allows to create a new document by starting from the content of an existing document in the repository. This is the same as the "duplicate document" functionality that is available in the Daisy Wiki.

The ID of the template document needs to be specified in a parameter called template. Branch and language can optionally be specified, they default to those of the current site.

http://localhost:8888/daisy/<sitename>/new/edit?template=123&branch=main&language=default
5.1.12.6.5 Creating a new document variant

The cases described until now were about creating entirely new documents. Here we look at how to open the editor to add a new variant to an existing document.

The following request parameters need to be specified, all required:

The branches and languages can be specified by name or ID.

So an example URL could be:

http://localhost:8888/daisy/<sitename>/new/edit?variantOf=123&startBranch=main&startLanguage=en&newBranch=main&newLanguage=fr
5.1.12.6.6 Creating a new document with custom initialisation

This allows to open the editor with certain data already in the new document. It works as follows:

This requires that you create a Daisy Wiki extension.

Here is an example:

cocoon.load("resource://org/outerj/daisy/frontend/util/daisy-util.js");

function makeNewDocument() {
    var daisy = new Daisy();
    var repo = daisy.getRepository();

    // Create the document
    // The parameters are the document name and the document type name
    var newDoc = repo.createDocument("A new document", "SimpleDocument");

    // Set some initial content in a part
    // The part content must be given as a byte array
    var initialContent = new java.lang.String("<html><body><p>Type something here."
                            + "</p></body></html>").getBytes("UTF-8");
    newDoc.setPart("SimpleDocumentContent", "text/xml", initialContent);

    // Add the document to the site's collection
    var siteCollection = repo.getCollectionManager().getCollection(
            daisy.getSiteConf().getCollectionId(), false);
    newDoc.addToCollection(siteCollection);

    // Store the document in a request attribute. The request attribute can have any name
    cocoon.request.setAttribute("myDoc", newDoc);

    // Switch to the editor
    var url = "cocoon:/" + daisy.getDaisyCocoonPath() + "/" + daisy.getSiteConf().getName()
              + "/new/edit?templateDocument=myDoc&startWithGet=true";
    cocoon.redirectTo(url);
}

For those not familiar with extensions yet, lets make the example complete.

Create a subdirectory in <wikidata dir>/sites/cocoon, for example called 'newtest'. Then save the above script in a file test.js in the newtest directory. Also in the newtest directory, create a file sitemap.xmap with the following content:

<?xml version="1.0"?>
<map:sitemap xmlns:map="http://apache.org/cocoon/sitemap/1.0">

  <map:components>
  </map:components>

  <map:views>
  </map:views>

  <map:resources>
  </map:resources>

  <map:flow language="javascript">
    <map:script src="test.js"/>
  </map:flow>

  <map:pipelines>

   <map:pipeline>
     <map:match pattern="newtest">
       <map:call function="makeNewDocument"/>
     </map:match>
   </map:pipeline>

 </map:pipelines>

</map:sitemap>

You can now call this example by surfing to the following URL:

http://localhost:8888/daisy/<sitename>/ext/newtest/newtest

(replace <sitename> and other stuff as appropriate. The first 'newtest' corresponds to the name of the 'newtest' directory, the second 'newtest' is the pattern matched in the sitemap.xmap)

5.1.12.7 Samples

5.1.12.7.1 Daisy Wiki extension sample: publish document

This sample shows how to publish just one document.

To use, just surf to an URL like:

http://localhost:8888/daisy/<sitename>/ext/publishdoc/<documentId>

in which you need to replace <sitename> with the name of your site and <documentId> with the ID of a document.

This simple example can be useful if you want to integrate a "published document" into an external system. Compared to retrieving a document from the default Daisy URLs, this example will not do any redirecting based on the navigation tree, and doesn't translate links in the document based on the navigation tree, which is the behaviour you'll usually prefer for this kind of applications. It's also easy to modify the XSL to create an XML envelope around the document containing any additional information you may need.

Building on this example, you could also do things like composing a page consisting of multiple published documents, adding one or more navigation trees, etc. All this just by extending the publisher request and doing some appropriate XSLing on the result.

5.1.12.7.2 Daisy Wiki extension sample: RSS include

This sample shows you how to include a live import of an Atom/RSS feed into a document.

Quite often, such feeds include escaped HTML (i.e. with < and > encoded as &lt; and &gt;), so we need some way to reparse that HTML into proper XML which can be passed through the Daisy publishing pipelines. Luckily, a Cocoon component exists for this exact purpose: the HTMLTransformer which is embedded in the Cocoon HTML Block. The HTML Block isn't included by default in the Daisy distribution, but the compiled jar can be found at the iBiblio Maven repository. Copy cocoon-html-2.1.7.jar into $DAISY_HOME/daisywiki/webapp/WEB-INF/lib/ and you should be set.

Then, you need to define an extension pipeline which transforms Atom markup into HTML for inclusion into a Daisy document.

Create a directory myrss inside <wikidata directory>/sites/cocoon and add this sitemap.xmap to it:

<?xml version="1.0"?>
<map:sitemap xmlns:map="http://apache.org/cocoon/sitemap/1.0">

  <map:components>
    <map:transformers>
      <map:transformer name="htmltransformer"
                       src="org.apache.cocoon.transformation.HTMLTransformer"/>
    </map:transformers>
  </map:components>
  <map:views>
  </map:views>
  <map:resources>
  </map:resources>

  <map:pipelines>

    <map:pipeline internal-only="true" type="noncaching">
      <map:match pattern="myrss">
        <map:generate src="/some/path/leading/upto/myatomfeed.xml"/>
        <!-- Thanks to Cocoon, this path could also be a URL.
             Keep in mind this URL will be accessed every time
             the including document is retrieved! -->
        <map:transform type="htmltransformer">
          <map:parameter name="tags" value="content"/>
        </map:transform>
        <map:transform type="xalan" src="atom2html.xsl"/>
        <map:serialize type="xml"/>
      </map:match>
    </map:pipeline>

  </map:pipelines>

</map:sitemap>

Next, add this XSL stylesheet (atom2html.xsl) to the myrss directory:

<?xml version="1.0"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
                xmlns:atom="http://purl.org/atom/ns#">
  
  <xsl:template match="/">
    <div>
      <xsl:for-each select="/atom:feed/atom:entry">
        <h2><xsl:value-of select="atom:title"/></h2>
        <xsl:copy-of select="atom:content/html/body/*"/>
      <p style="text-align: right; color: #999; font-size: 80%;">
        Originally blogged by <xsl:value-of select="atom:author/atom:name"/> on
        <a href="{atom:link/@href}"><xsl:value-of select="atom:issued"/></a>.
      </p>
      </xsl:for-each>
    </div>
  </xsl:template>

</xsl:stylesheet>

The last work is to include the snippet of generated HTML into a Daisy document. Add an "include"-style paragraph to the document content with this content:

cocoon:/ext/myrss/myrss

That's it! When you save the document, you'll see that the Atom feed is dynamically pulled into the webpage as a partial HTML document.

5.1.12.7.3 Daisy Wiki extension sample: guestbook

This sample illustrates using a form to collect data (based on Cocoon's form framework, CForms) and creating a document in the repository using the collected data. The theme of the sample is a "guestbook", though we don't want to publicise this as the correct way of building a guestbook.

This sample is not included in the by default deployed cross-site extensions, but must be manually 'installed' (copied) into a certain site, and requires some small customisation. See the directory <webapp>/daisy/ext-samples and the README.txt file over there.

5.1.12.7.4 Daisy Wiki extension sample: navigation aggregation

This example creates a page composed of all documents occurring in a navigation tree, in the order in which they occur in the navigation tree. It is also possible to limit the result to a certain subsection of the navigation tree.

To use, call an URL like:

http://localhost:8888/daisy/<sitename>/ext/navaggregator/navaggregator

in which you replace <sitename> by the name of your site.

To limit to a certain section of the navigation tree, add an activePath request parameter:

http://localhost:8888/daisy/<sitename>/ext/navaggregator/navaggregator?activePath=/abc/def

You can see the active path in the browser location bar when you move over nodes in a navigation tree, it is the part after the sitename.

The basic example is very simple, it could be extended do to things like:

All this isn't that particular difficult to realize either.

5.1.12.7.5 RSS

RSS feeds are provided in the Daisy Wiki as a cross-site extension, since there are many different feeds that make sense, and this easily allows to build your own if you don't like the ones provided by default.

The URLs for the default feeds provided by this extension are (replace <sitename> with the name of a site):

http://localhost:8888/daisy/<sitename>/ext/rss/minimal-rss.xml
http://localhost:8888/daisy/<sitename>/ext/rss/normal-rss.xml
http://localhost:8888/daisy/<sitename>/ext/rss/editors-rss.xml

The differences between these feeds are:

The following notes apply to all the feeds.

Non-cached feeds can be accessed with the following URLs:

http://localhost:8888/daisy/<sitename>/ext/rss/minimal-rss-direct.xml
http://localhost:8888/daisy/<sitename>/ext/rss/normal-rss-direct.xml
http://localhost:8888/daisy/<sitename>/ext/rss/editors-rss-direct.xml

To have RSS-links appear on the recent changes page, the RSS feeds need to be configured in the skinconf.xml. The default skinconf.xml contains the config for the above RSS feeds, so you can have a look there to see how its done.

5.1.13 RSS

RSS feeds are provided in the Daisy Wiki as a cross-site extension, since there are many different feeds that make sense, and this easily allows to build your own if you don't like the ones provided by default.

The URLs for the default feeds provided by this extension are (replace <sitename> with the name of a site):

http://localhost:8888/daisy/<sitename>/ext/rss/minimal-rss.xml
http://localhost:8888/daisy/<sitename>/ext/rss/normal-rss.xml
http://localhost:8888/daisy/<sitename>/ext/rss/editors-rss.xml

The differences between these feeds are:

The following notes apply to all the feeds.

Non-cached feeds can be accessed with the following URLs:

http://localhost:8888/daisy/<sitename>/ext/rss/minimal-rss-direct.xml
http://localhost:8888/daisy/<sitename>/ext/rss/normal-rss-direct.xml
http://localhost:8888/daisy/<sitename>/ext/rss/editors-rss-direct.xml

To have RSS-links appear on the recent changes page, the RSS feeds need to be configured in the skinconf.xml. The default skinconf.xml contains the config for the above RSS feeds, so you can have a look there to see how its done.

5.1.14 Part Editors

5.1.14.1 Introduction

Starting with Daisy 1.4, the document editor has been changed to make it easy to insert custom editors for parts.

5.1.14.2 Configuration

The rules for detecting which part editor to use are as follows:

To use a custom editor for a certain part, a file called <partTypeName>.xml should be created in the following directory: 

<wikidata directory>/resources/parteditors/

The file should contain some XML like the following:

<?xml version="1.0"?>
<partEditor xmlns="http://outerx.org/daisy/1.0#parteditor"
        class="...">
  <properties>
    <entry key="...">...</entry>
  </properties>
</partEditor>

The class property of the partEditor element should contain the class name of a class implementing the following interface:

org.outerj.daisy.frontend.editor.PartEditorFactory

The properties element is optional and can be used to configure the part editor.

Besides the default editors, there is one example editor included in Daisy: a plain text editor. Suppose you have a part called MyPlainText and you want to use this editor for it, you need to create a file called MyPlainText.xml in the above mentioned directory, and put the following XML in it:

<?xml version="1.0"?>
<partEditor xmlns="http://outerx.org/daisy/1.0#parteditor"
        class="org.outerj.daisy.frontend.editor.PlainTextPartEditor$Factory">
</partEditor>

Once you save this file, the editor will become immediately active (no need to restart the Wiki). If you do not see the editor, check the cocoon error log.

5.1.14.3 Implementation info

A part editor is implemented by means of a CForm (Cocoon Form). This consists of a form definition and a form template. To fit into Daisy's document editor framework, there are two things you need to pay attention to:

Besides the form definition and template, you need to implement the interface org.outerj.daisy.frontend.PartEditor (and corresponding PartEditorFactory). This interface defines methods that will be called for constructing, loading and saving of the form.

To learn more, it is best to explore the sources of the default Daisy part editors.

5.1.15 Internationalisation

5.1.15.1 Introduction

We welcome translations to new languages, or improvements to the currently available languages. Send translations to the mailing list, or attach them to an issue in Jira. Make sure you use UTF-8 encoding in your files (except for the .properties files -- see below). Also, check on the mailing list if anybody else is working on the same language as you.

Daisy and the Daisy Wiki can be translated to different languages. For this purpose, text to be displayed to users is retrieved from a language-specific file, usually called a resource bundle.

5.1.15.2 Different types of resource bundles

There are three different types of resource bundles used by Daisy:

When editing resource bundles, take care of the encoding of the files, as explained below.

5.1.15.2.1 Encoding of the XML and .js files

We prefer to keep all the XML and .js (javascript) files UTF-8 encoded, for all languages. Therefore, take care to use the correct encoding when opening and saving files in your text editor.

5.1.15.2.2 Encoding of the .properties files

The ".properties" files should always be ISO-8859-1 encoded. Characters not supported by ISO-8859-1 can be inserted using Java unicode escapes. A Java unicode escape takes the form "\uHHHH" in which the HHHH are hexadecimal digits specifying the unicode character. Usually you don't want to edit this by hand, the unix recode utility can be handy here:

  1. (if it is an existing file) do:
     recode JAVA..UTF-8 somefile.properties
  2. then open the file in an editor using UTF-8 as encoding
  3. then convert back:
    recode UTF-8..JAVA somefile.properties

5.1.15.3 Overview of the resource bundle files

Below is an overview of the translatable resources that currently exists. The given directory paths refer to locations inside the source tree, in the binary distribution these locations are a bit different (and the .property files are bundled inside the jar files). See further on this page for practical instructions.

To make the new language appear on the locale selection page, its ISO language code should be added to the following file:

applications/daisywiki/frontend/src/cocoon/webapp/daisy/resources/conf/locales.txt

5.1.15.4 Windows installer

All installation messages of the windows installer are kept inside on single file, which also acts as template for any new languages to be added:

distro/windows-installer/installer/lang/english.nsh

The NSIS installer does not support Unicode yet, it assumes no encoding and just copies the bytes as is. Therefore the encoding used is mentioned in the comment section in the header of the .nsh language file. In order to display the messages correctly, the default language for non-Unicode applications in the language control panel of your Windows installation has to be set appropriately.

5.1.15.5 Making or improving a translation

Install the subversion client if you don't have it already.

Check out the Daisy source tree:

svn co http://svn.cocoondev.org/repos/daisy/trunk/daisy

After editing the files, create a patch by executing the following command in the root of the Daisy source tree:

svn diff > mypatch.txt

and send the mypatch.txt file to the mailing list.

5.1.16 User self-registration

People can register themselves as users in the Daisy Wiki. By default, they get assigned the "guest" role. For the default ACL (access control list) that ships with Daisy, this means a registered user will still not be able to edit documents. However, the user will be able to subscribe to notification emails and to add comments to documents.

If you want self-registered users to have a different role(s) assigned after registration, you can configure this in the following file:

<wikidata directory>/daisy.xconf

In that file, search for the string UserRegistrar. On the next few lines you'll see this:

    <!-- The names of the roles to which the user should be assigned -->
    <roles>
      <role>guest</role>
    </roles>
    <!-- The default role for the user, should be one of those mentioned in the roles list. -->
    <defaultRole>guest</defaultRole>

which you can adjust according to your wishes.

To have this change take effect, you need to restart the Daisy Wiki.

5.1.17 Live and staging view

By default, the Daisy Wiki shows the live version of each document, if you want to see other versions you need to explicitly go look at them.

The live/staging switch allows to make that the last version of each document is displayed by default. Or from another point of view, it shows how the Wiki would look like if the last versions of all documents were live.

The live/staging applies to:

Since the fulltext index is only maintained for the live content, the fulltext search feature doesn't change when switching to staging mode.

The live/staging mode is stored in the user's session, in other words it applies to the whole Daisy Wiki (all sites), and is not remembered after logging out or closing the browser.

More technically, the live/staging switch is implemented by setting the versionMode attribute in the publisher request.

In extensions, you can find out the current mode using the daisy.getVersionMode() method of the Daisy flowscript API.

5.2 Technical

5.2.1 Document Publishing

This document will describe the process of how a document gets published by Daisy. What follows below are some first rough notes.

See also the Cocoon GT presentation on this subject.

The publishing process is split between a "Publisher" extension component running in the repository server and the frontend running on Cocoon.

The goal of the Publisher component is to gather all information needed to publish a document and return the result to Cocoon as an XML document. This avoids a lot of relatively expensive remote calls that would otherwise need to happen. More specifically, here's what the Publisher component does:

Included documents are not directly inserted at the location of inclusion, rather they will added "side by side" to the output. This is to allow to execute different XSLs on them on the Cocoon side.

All this processing is done streaming (SAX). (with the exception of some parsing results that are first pushed into buffers -- SaxBuffer instances -- to gracefully handle parsing exceptions)

Cocoon then performs the following tasks:

With exception of the XSLTs (for which we keep the input as small as possible) and the temporary buffers containing the (styled) documents, all processing again happens in a SAX-streaming manner.

6 Book publishing

6.1 Daisy Books Overview

6.1.1 Introduction

The purpose of Daisy Books is to publish a set of aggregated Daisy documents as one whole, thus like a book or manual. Books can be published in various forms, including PDF and HTML (as one page or chunked into multiple pages).

Technically, Daisy Books is part of (= integrated in) the Daisy Wiki, though the process of publishing a book is completely unrelated to that of publishing a Wiki page.

Daisy Books offers a lot of common features required for books, including:

Books are published 'off-line' (in batch). This means a user will trigger the process to aggregate and format the Daisy documents into an actual book. The resulting book can then be consulted by other users (the book readers).

6.1.2 Terminology

Let us first agree on some terminology. Here we only give some very general definitions, which are described in more detail later on.

Book definition. A book definition is a Daisy document that defines the structure and content of the book (which documents to include), it defines general book metadata, and it defines default parameters for the publication of the book.

Publication process. A publication process generates an actual book (a book instance) based on a book definition, according to one ore more publication types.

Book instance. A book instance, or simply book, is the result of running a book publication process. It is a collection of files (organised in directories) among which the published PDF file and/or HTML files, but also a snapshot of the data which is included in the book, intermediary book production files, a book publication log and a link errors log. Book instances are created and managed by the book store.

Book store. The book store manages book instances. In other words, the book store is where published books are stored.

Publication type. A publication type defines one way of how a book can be published (for example "PDF" or "Chunked HTML"). Publication types can be customised to a certain extent through properties (for example, to define the depth of the generated TOC), though for most layout changes you will usually create your own publication types (starting from the default ones).

Publication output. A publication output, or simply publication, is the result of publishing a book according to a publication type (thus a set of files).

6.1.3 The book publication process

The process of publishing a book consists of two main parts:

  1. Book data retrieval: retrieves, from the repository server, all the data (documents and images) needed for inclusion in the book. This data is stored as part of the book instance.
  2. Publication type specific publishing: performs the publishing according to a publication type (this step is performed multiple times if multiple publication types were selected)

These two parts are now described in some more detail.

6.1.3.1 Book data retrieval

The needed book data is retrieved and stored in the book instance. There are multiple reasons for doing this:

The documents are retrieved using the Publisher component in the form of 'prepared documents'.

While retrieving the data, a list is compiled of all retrieved documents with their exact version retrieved. This information can be used later on to check what documents were updated since the publishing of a book.

6.1.3.2 Publication type specific publishing

This process is defined by the publication type, but usually follows these steps:

6.2 Creating a book

The only thing required to publish a book is a book definition.

6.2.1 Quickly trying it out

To create (publish) a book based on existing content:

6.2.2 Creating a new book from scratch

Usually, you will simply set up a Daisy Wiki site for the purpose of creating/editing the book content. However, instead of using a navigation tree definition, you will use a book definition.

Assuming you have no existing content yet, these are the steps to set up a new book (described in more detail below):

  1. Creating a collection which will contain the documents for the books
  2. Create a first document to include in the book
  3. Create a book definition, for now just containing a reference to the document created in the previous step
  4. Define a new Wiki site, with as home page the created document and as navigation tree the book definition.
  5. Verify the site works
  6. Try to publish the book

You will need Administrator access (to create the collection) and access to the machine on which the Daisy Wiki is running (to define the new site, there is no GUI for this yet).

Now lets run over these steps in more detail.

6.2.2.1 Creating a collection

In the Daisy Wiki, go to the Administration pages.

The Administration link is only visible if you have the "Administrator" role selected.

Over there, create a new collection (this should point itself out). Write down the ID of the created collection, we will need it later on.

6.2.2.2 Create a first document

In the Daisy Wiki, browse to some existing site, and create a new document (usually of type "Simple Document"). Type some initial dummy text in the document (like "hello world!"). Switch to the collections tab and add the document to the newly created collection, and remove it from any other collections.

Save the document. Write down the ID of the document (which you can see in the URL or else by selecting the "Document Info" link).

6.2.2.3 Create a book definition

Again in the Daisy Wiki, create a new document but this time of type "Book Definition". As name for the document, enter something like "First Book" or an appropriate name.

In the first part ("Book Definition Description"), click on the icon to insert a new section node, and fill in the ID of the document created in the previous step. Alternatively, if the GUI editor does not work in your browser, you can copy and paste the following XML:

<?xml version='1.0' ?>
<b:book  xmlns:b="http://outerx.org/daisy/1.0#bookdef">
  <b:content>
    <b:section documentId="xyz"/>
  </b:content>
</b:book>

Replace the xyz in the above XML with the ID of the document created in the previous step.

Then switch to the part "Book Metadata", and edit the title of the book to something appropriate.

Now save this document, and again write down its ID.

6.2.2.4 Define a new Wiki site

On the computer on which the Daisy Wiki is installed, go to the following directory:

<wikidata directory>/sites

In there, create a new subdirectory, with some name of your choice (without spaces). For example, "firstbook".

In this newly created directory, create a file called siteconf.xml with the following content:

<siteconf xmlns="http://outerx.org/daisy/1.0#siteconf">
  <title>First Book</title>
  <description>My first book</description>
  <skin>default</skin>
  <navigationDocId>book_def_id</navigationDocId>
  <homepageDocId>first_doc_id</homepageDocId>
  <collectionId>collection_id</collectionId>
  <contextualizedTree>true</contextualizedTree>
  <branch>main</branch>
  <language>default</language>
  <defaultDocumentType>SimpleDocument</defaultDocumentType>
  <newVersionStateDefault>publish</newVersionStateDefault>
  <locking>
    <automatic lockType='pessimistic' defaultTime='15' autoExtend='true'/>
  </locking>
</siteconf>

In this XML, replace the highlighted strings "book_def_id", "first_doc_id" and "collection_id" with the correct IDs of the items created in the previous steps. Then save this file.

6.2.2.5 Verify the site works

Go to the site index page (in the Daisy Wiki, the "Daisy Home" link in the right top). You should see the new site ("First Book") appear in the list. If you do not see it, wait ten seconds and refresh the page (it can take up to 10 seconds before the new site gets detected). If it does still not appear, it is usually because there is some error in the site definition. In that case, check the following log file:

<wikidata directory>/logs/error.log

Another reason a site might not appear is because you don't have read access to its home page. However, since you just created these documents, this is unlikely.

Click on the site link, you should see the usual layout with the navigation tree containing your first document.

6.2.2.6 Try to publish the book

Go to the following URL:

http://localhost:8888/daisy/books

(or equivalent according to your setup)

Choose the link "Publish a book". You get a page that shows the available book definitions. At this point, probably only the one you just created. Click on the "Publish" link next to it.

You can consult your published books at any later time by surfing again to the books URL:

http://localhost:8888/daisy/books

There you can also adjust the ACL so that others can see the book, or delete it.

6.2.2.7 Notes

6.2.3 Converting an existing Daisy Wiki site to a book

If you have a site which you would like to publish as a book, and the navigation tree of the site represents the content of the book, you can easily transform the existing navigation tree to a book definition using an XSL, as follows:

Optionally, you can edit the siteconf.xml of the Daisy Wiki site and change the content of the <navigationDocId> element to point directly to the book definition (the book definition can serve as a replacement of the navigation tree).

Now you can publish your book as described above under the heading "Try to publish the book".

6.3 Technical guide

6.3.1 Book Definition

This document describes the purpose and content o