Table of Contents

How to configure InDefero with monotone as SCM backend

Plugin SyncMonotone by Thomas Keller (me@thomaskeller.biz)

The SyncMonotone plugin allow the direct creation and synchronisation of monotone repositories with the InDefero database. It has been built to work together with monotone's "super server" usher, which is used to control several repositories at once, acts as proxy and single entrance.

Prerequisites

a unixoid operating system
monotone >= 0.99.1
for a proxy setup with usher:
- boost headers (for usher compilation)
- a current version of usher
- a daemonizer, like supervise

Installation of monotone

If you install monotone from a distribution package, ensure you do not install and / or activate the server component. We just need a plain client installation which usually consists only of the mtn binary and a few docs.

If you install monotone from source (http://monotone.ca/downloads.php), please follow the INSTALL document which comes with the software. It contains detailed instructions, including all needed dependencies.

Choose your indefero setup

The monotone plugin can be used in several different ways:

One database for everything. This is the easiest setup and of possible use in case you do not want indefero to manage the access to your project. Your idf.php should look like this:
```
    $ cat idf.php
    ...
    $cfg['mtn_path'] = 'mtn';
    $cfg['mtn_opts'] = array('--no-workspace', '--no-standard-rcfiles');
    $cfg['mtn_repositories'] = '/home/monotone/all_projects.mtn';
    $cfg['mtn_remote_url'] = 'ssh://monotone@my.server.com:~all_projects.mtn';
    $cfg['mtn_db_access'] = 'local';
    ...
```
Pro:
- easy to setup and to manage
Con:
- you need to give committers SSH access to your machine
- database lock problem: the database from which indefero reads its data might be locked in case a user syncs at the very moment via SSH
One database for every project. Similar to the above setup, but this time you use the '%s' placeholder which is replaced with the short name of the indefero project:
```
    $ cat idf.php
    ...
    $cfg['mtn_path'] = 'mtn';
    $cfg['mtn_opts'] = array('--no-workspace', '--no-standard-rcfiles');
    $cfg['mtn_repositories'] = '/home/monotone/%s.mtn';
    $cfg['mtn_remote_url'] = 'ssh://monotone@my.server.com:~%s.mtn';
    $cfg['mtn_db_access'] = 'local';
    ...
```
The same pro's and con's apply. Additionally you have to be careful about not giving people physical read/write access of another project's database.

Furthermore, if you do not want to use ssh, but netsync transport, each project's database must be served over a separate port.
One database for every project, all managed with usher. This is the recommended setup for a mid-size forge setup. The remaining part of this document will describe the process to set this up in detail.

Pro:
- access rights can be granted per project and are automatically managed by indefero, just like the user's public monotone keys
- no database locking issues
- one public server running on the one well-known port
Con:
- harder to setup

Installation and configuration of usher

Clone usher's monotone repository:

    $ mtn clone "mtn://monotone.ca?net.venge.monotone.contrib.usher"

Compile usher:

    $ autoreconf -i
    $ ./configure && make
    $ sudo make install

This installs the usher binary in $prefix/bin.

Create a new usher user:

    $ adduser --system --disabled-login --home /var/lib/usher usher

Create the basic usher setup:

    $ cd /var/lib/usher
    $ mkdir projects logs
    $ cat > usher.conf
     userpass "admin" "<secret-password>"
    adminaddr "127.0.0.1:12345"
       logdir "log"
    ^D
    $ chmod 600 usher.conf

Your indefero www user needs later write access to usher.conf and projects/. There are two ways of setting this up:

Make the usher user the web user, for example via Apache's suexec

Use acls, like this:

$ setfacl -m u:www:rw usher.conf
$ setfacl -m d:u:www:rwx projects/

Wrap a daemonizer around usher, for example supervise from daemontools (http://cr.yp.to/damontools.html):

    $ cat > run
    #!/bin/sh
    cd /var/lib/usher
    exec 2>&1
    exec \
        setuidgid usher \
        usher usher.conf
    ^D

The service can now be started through supervise:

    $ supervise /var/lib/usher

Configuration of indefero

Based on the above setup, the configuration in src/IDF/conf/idf.php should look like this:

    $ cat idf.php
    ...
    $cfg['mtn_path'] = 'mtn';
    $cfg['mtn_opts'] = array('--no-workspace', '--no-standard-rcfiles');
    $cfg['mtn_repositories'] = '/var/lib/usher/projects/%s/';
    $cfg['mtn_remote_url'] = 'mtn://my.server.com/%s';
    $cfg['mtn_db_access'] = 'remote';
    $cfg['mtn_usher_conf'] = '/var/lib/usher/usher.conf';
    ...

The %s placeholders are automatically replaced by the name of the indefero project. The plugin assumes that every project is separated by a distinct server name in the monotone URL (hence the use of /%s), so if a user calls

    $ mtn sync mtn://my.server.com/project1

then the database / repository of the indefero project1 is used. Note that 'mtn_remote_url' is also used as internal URI to query the data for indefero's source view, so it must be a valid host!

Usher also allows the identification of a project repository by hostname, which would allow an URL template like mtn://%s.my.server.com, however the plugin does not write out the configuration which is needed for this yet.

For even more advanced setups, usher can also be used to forward sync requests to other remote servers for load balancing, please consult the README file for more information.

Security and remote access

Indefero distinguishs between public and private projects and so does the monotone plugin.

Public projects can be pulled by everybody and pushed by team members or additional invited people. Remote command execution is enabled, but only for read-only commands.

Remote commands can be helpful for a user or a 3rd party tool (like mtn-browse or guitone) to browse the database contents remotely without having to pull everything in first instance.

Private projects on the other hand can only be synced by team members or additional invited people. Remote command execution is disabled by default. If you want to enable that, simply put the keys of the users you want to give access to in your project's remote-automate-permissions file. In the future this plugin might handle this file just as it handles read-permissions and write-permissions.

Notifications

If you have successfully set up your monotone instance, you probably want to notify 3rd party systems for incoming changes or simply mirror them somewhere else for backup purposes. The monotone source tree already comes with many example scripts and hooks which serve these purposes, after only little additional configuration.

The usher/indefero-controlled setup automatically looks for *.lua files in a directory called hooks.d right under the project's base directory (configured via $cfg['mtn_repositories']) and this is the ideal place to put or link these additional lua sources.

Custom project configurations and templates

If a new project is created in IDF, the SyncMonotone plugin creates a new configuration tree for the project into the project's configuration directory, determined by $cfg['mtn_repositories']. IDF ships with the minimum set of files for this configuration tree and sets up everything automatically for you.

Even more, most of the configuration files from the newly created tree are only symlinked to the original configuration directory which is configurable via $cfg['mtn_confdir'] and defaults to src/IDF/Plugin/SyncMonotone/. This has the advantage that your standard IDF setup automatically receives updates to existing (symlinked) configuration files as soon as you update to a newer version.

You could, however, also choose to place the directory tree somewhere else and adapt the contents of the individual files yourself, so these changes get automatically applied to all new projects you create. You could even go so far and add new files to the tree and let them be processed automatically just as the basic files! All you need to do is to copy your files and / or directories underknees your $cfg['mtn_confdir'] and add their relative paths to $cfg['mtn_confdir_extra'].

By convention, all entries which end with a slash are considered directories, so mkdir(1) is issued for these entries, all files which do not end up with ".in" are considered to be static script files which are just symlinked from the basic configuration dir and all entries ending on ".in" are considered configuration files or templates, which are copied over to the project's configuration tree and which get some basic project-specific values replaced.

The following placeholders are currently recognized and replaced for these files:

%%PROJECT%% - the name of the created project
%%MTNPOSTPUSH%% - the absolute path to the mtn-post-push script
%%MTNCLIENTKEY%% - the public key hash of the key which is used by IDF to authenticate remote stdio access

Thats it - I hope you find it useful :)

Q&A

After I created a new project, IDF throws an exception and tells me that it couldn't save the membership data with a cryptic error message. Whats wrong?

Multiple issues could cause that. If you've set up usher, make sure the usher can fork your database at all and look out for specific errors in the log file of your project. If you stumble upon permission issues, ensure that the user who runs the usher can access all files in your project's configuration directory, including symlinked files.

I pushed a branch to my server, but it does not show up in IDF. Whats wrong?

Check if the heads of your branch are not suspended, i.e. do not carry a suspend certificate. This usually hides the branch and all of its history from monotone's eyes and therefor also from indefero. You can either choose to "unsuspend" the branch simply by committing and pushing another head or by letting monotone ignore all suspend certs. For the latter, its usually enough to add --ignore-suspend-certs to the list of options in $cfg['mtn_opts'].

I want to display another default branch when I click the "Source" tab. How can I do that?

Let the forge admin know the new master branch for your project. He is able to change that quickly. Depending on the backend / server setup this might also require some changes in the usher configuration, but only if usher recognizes and proxies your database on a branch name level.

Indefero

InstallationScmMonotone