Indefero

Indefero Commit Details


Date:2010-09-16 20:11:36 (15 years 2 months ago)
Author:Thomas Keller
Branch:develop, feature-issue_links, feature.better-home, feature.content-md5, feature.diff-whitespace, feature.download-md5, feature.issue-links, feature.issue-of-others, feature.issue-summary, feature.search-filter, feature.webrepos, feature.wiki-default-page, master, release-1.1, release-1.2, release-1.3
Commit:617589f41b41d594249d0697688d8b7d0db1166f
Parents: 50638c768f302f278c94583fac36fc4818ff47d6
Message:Reorganize and expand the help of the monotone plugin. Make the commentary in idf.php-dist less verbose.
Changes:

File differences

doc/readme-monotone.mdtext
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
# monotone implementation notes␊
␊
## general␊
␊
This version of indefero contains an implementation of the monotone ␊
automation interface. It needs at least monotone version 0.99 ␊
(interface version 13.0) or newer.␊
␊
To set up a new IDF project with monotone quickly, all you need to do␊
is to create a new monotone database with␊
␊
$ mtn db init -d project.mtn␊
␊
in the configured repository path `$cfg['mtn_repositories']` and␊
configure `$cfg['mtn_db_access']` to "local".␊
␊
To have a really workable setup, this database needs an initial commit␊
on the configured master branch of the project. This can be done easily␊
with␊
␊
$ mkdir tmp && touch tmp/remove_me␊
$ mtn import -d project.mtn -b master.branch.name \␊
-m "initial commit" tmp␊
$ rm -rf tmp␊
␊
Its expected that more scripts arrive soon to automate this and other␊
tasks in the future for (multi)forge setups.␊
␊
## current state / internals␊
␊
The implementation should be fairly stable and fast, though some␊
information, such as individual file sizes or last change information,␊
won't scale well with the tree size. Its expected that the mtn␊
automation interface improves in this area in the future and that␊
these parts can then be rewritten with speed in mind.␊
␊
As the idf.conf-dist explains more in detail, different access patterns␊
are possible to retrieve changeset data from monotone. Please refer␊
to the documentation there for more information.␊
␊
## indefero critique:␊
␊
It was not always 100% clear what some of the abstract SCM API method␊
wanted in return. While it helped a lot to have prior art in form of the␊
SVN and git implementation, the documentation of the abstract IDF_Scm␊
should probably still be improved.␊
␊
Since branch and tag names can be of arbitrary size, it was not possible␊
to display them completely in the default layout. This might be a problem␊
in other SCMs as well, in particular for the monotone implementation I␊
introduced a special filter, called "IDF_Views_Source_ShortenString".␊
␊
The API methods getPathInfo() and getTree() return similar VCS "objects"␊
which unfortunately do not have a well-defined structure - this should␊
probably addressed in future indefero releases.␊
␊
While the returned objects from getTree() contain all the needed␊
information, indefero doesn't seem to use them to sort the output␊
f.e. alphabetically or in such a way that directories are outputted␊
before files. It was unclear if the SCM implementor should do this␊
task or not and what the admired default sorting should be.␊
␊
doc/syncmonotone.mdtext
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
# Plugin SyncMonotone by Thomas Keller (me@thomaskeller.biz)␊
␊
The SyncMonotone plugin allow the direct creation and synchronisation of␊
monotone repositories with the InDefero database. It has been built to␊
work together with monotone's "super server" usher, which is used to control␊
several repositories at once, acts as proxy and single entrance.␊
␊
## Prerequisites␊
␊
* a unixoid operating system␊
* monotone >= 0.99␊
* for a proxy setup with usher:␊
* boost headers (for usher compilation)␊
* a current version of usher␊
* a daemonizer, like supervise␊
␊
## Installation of monotone␊
␊
If you install monotone from a distribution package, ensure you do not␊
install and / or activate the server component. We just need a plain␊
client installation which usually consists only of the `mtn` binary and␊
a few docs.␊
␊
If you install monotone from source (<http://monotone.ca/downloads.php>),␊
please follow the `INSTALL` document which comes with the software.␊
It contains detailed instructions, including all needed dependencies.␊
␊
## Choose your indefero setup␊
␊
The monotone plugin can be used in several different ways:␊
␊
1. One database for everything. This is the easiest setup and of possible␊
use in case you do not want indefero to manage the access to your project.␊
Your `idf.php` should look like this:␊
␊
$ cat idf.php␊
...␊
$cfg['mtn_path'] = 'mtn';␊
$cfg['mtn_opts'] = array('--no-workspace', '--no-standard-rcfiles');␊
$cfg['mtn_repositories'] = '/home/monotone/all_projects.mtn';␊
$cfg['mtn_remote_url'] = 'ssh://monotone@my.server.com:~all_projects.mtn';␊
$cfg['mtn_db_access'] = 'local';␊
...␊
␊
Pro:␊
* easy to setup and to manage␊
␊
Con:␊
* you need to give committers SSH access to your machine␊
* database lock problem: the database from which␊
indefero reads its data might be locked in case a user␊
syncs at the very moment via SSH␊
␊
2. One database for every project. Similar to the above setup, but this␊
time you use the '%s' placeholder which is replaced with the short name␊
of the indefero project:␊
␊
$ cat idf.php␊
...␊
$cfg['mtn_path'] = 'mtn';␊
$cfg['mtn_opts'] = array('--no-workspace', '--no-standard-rcfiles');␊
$cfg['mtn_repositories'] = '/home/monotone/%s.mtn';␊
$cfg['mtn_remote_url'] = 'ssh://monotone@my.server.com:~%s.mtn';␊
$cfg['mtn_db_access'] = 'local';␊
...␊
␊
The same pro's and con's apply. Additionally you have to be careful about␊
not giving people physical read/write access of another project's database.␊
␊
Furthermore, if you do not want to use `ssh`, but `netsync` transport,␊
each project's database must be served over a separate port.␊
␊
3. One database for every project, all managed with usher. This is the␊
recommended setup for a mid-size forge setup. The remaining part of this␊
document will describe the process to set this up in detail.␊
␊
Pro:␊
* access rights can be granted per project and are automatically␊
managed by indefero, just like the user's public monotone keys␊
* no database locking issues␊
* one public server running on the one well-known port␊
␊
Con:␊
* harder to setup␊
␊
## Installation and configuration of usher␊
␊
1. Clone usher's monotone repository:␊
␊
$ mtn clone "mtn://monotone.ca?net.venge.monotone.contrib.usher"␊
␊
2. Compile usher:␊
␊
$ autoreconf -i␊
$ ./configure && make␊
$ sudo make install␊
␊
This installs the usher binary in $prefix/bin.␊
␊
3. Create a new usher user:␊
␊
$ adduser --system --disabled-login --home /var/lib/usher usher␊
␊
4. Create the basic usher setup:␊
␊
$ cd /var/lib/usher␊
$ mkdir projects logs␊
$ cat > usher.conf␊
userpass "admin" "<secret-password>"␊
adminaddr "127.0.0.1:12345"␊
logdir "log"␊
^D␊
$ chmod 600 usher.conf␊
␊
Your indefero www user needs later write access to `usher.conf` and␊
`projects/`. There are two ways of setting this up:␊
␊
* Make the usher user the web user, for example via Apache's `suexec`␊
* Use acls, like this:␊
␊
$ setfacl -m u:www:rw usher.conf␊
$ setfacl -m d:u:www:rwx projects/␊
␊
5. Wrap a daemonizer around usher, for example supervise from daemontools␊
(<http://cr.yp.to/damontools.html>):␊
␊
$ cat > run␊
#!/bin/sh␊
cd /var/lib/usher␊
exec 2>&1␊
exec \␊
setuidgid usher \␊
usher usher.conf␊
^D␊
␊
The service can now be started through supervise:␊
␊
$ supervise /var/lib/usher␊
␊
## Configuration of indefero␊
␊
Based on the above setup, the configuration in `src/IDF/conf/idf.php` should␊
look like this:␊
␊
$ cat idf.php␊
...␊
$cfg['mtn_path'] = 'mtn';␊
$cfg['mtn_opts'] = array('--no-workspace', '--no-standard-rcfiles');␊
$cfg['mtn_repositories'] = '/var/lib/usher/projects/%s/';␊
$cfg['mtn_remote_url'] = 'mtn://my.server.com/%s';␊
$cfg['mtn_db_access'] = 'remote';␊
$cfg['mtn_remote_auth'] = true;␊
$cfg['mtn_usher_conf'] = '/var/lib/usher/usher.conf';␊
...␊
␊
The `%s` placeholders are automatically replaced by the name of the␊
indefero project. The plugin assumes that every project is separated␊
by a distinct server name in the monotone URL (hence the use of `/%s`),␊
so if a user calls␊
␊
$ mtn sync mtn://my.server.com/project1␊
␊
then the database / repository of the indefero `project1` is used.␊
Note that 'mtn_remote_url' is also used as internal URI to query the data␊
for indefero's source view, so it *must* be a valid host!␊
␊
Usher also allows the identification of a project repository by hostname,␊
which would allow an URL template like `mtn://%s.my.server.com`, however␊
the plugin does not write out the configuration which is needed for this␊
yet.␊
␊
For even more advanced setups, usher can also be used to forward sync␊
requests to other remote servers for load balancing, please consult the␊
README file for more information.␊
␊
src/IDF/conf/idf.php-dist
7373
7474
7575
76
76
77
78
79
80
81
7782
83
7884
7985
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
86
87
88
14589
90
91
92
14693
147
94
14895
14996
150
151
152
153
154
155
156
157
158
159
97
98
99
100
160101
161102
162103
......
167108
168109
169110
170
171
172
173
174
111
112
113
114
115
175116
176117
177
178118
179119
180120
$cfg['svn_repositories'] = 'file:///home/svn/repositories/%s';␊
$cfg['svn_remote_url'] = 'http://localhost/svn/%s';␊
␊
# Path to the monotone binary (you need mtn 0.99 or newer)␊
#␊
# You can setup monotone for use with indefero in several ways.␊
# Please look into doc/syncmonotone.mdtext for more information.␊
#␊
␊
# Path to the monotone binary␊
$cfg['mtn_path'] = 'mtn';␊
␊
# Additional options for the started monotone process␊
$cfg['mtn_opts'] = array('--no-workspace', '--no-standard-rcfiles');␊
#␊
# You can setup monotone for use with indefero in several ways. The␊
# two most-used should be:␊
#␊
# 1) One database for everything:␊
#␊
# Set 'mtn_repositories' below to a fixed database path, such as␊
# '/home/mtn/repositories/all_projects.mtn'␊
#␊
# Pro: - easy to setup and to manage␊
# Con: - while read access can be configured per-branch,␊
# granting write access rights to a user means that␊
# he can write anything in the global database␊
# - database lock problem: the database from which␊
# indefero reads its data cannot be used to serve the␊
# contents to the users, as the serve process locks␊
# the database␊
#␊
# 2) One database for every project with 'usher':␊
#␊
# Download and configure 'usher'␊
# (mtn clone mtn://monotone.ca?net.venge.monotone.contrib.usher)␊
# which acts as proxy in front of all single project databases.␊
# Create a basic configuration file for it and add a secret admin␊
# username and password. Finally, point the below variable␊
# 'mtn_usher_conf' to this configuration file.␊
#␊
# Then set 'mtn_remote_url' below to a string which matches your setup.␊
# Again, the '%s' placeholder will be expanded to the project's␊
# short name. Note that 'mtn_remote_url' is used as internal␊
# URI (to access the data for indefero) as well as external URI␊
# (for end users) at the same time. 'mtn_repositories' should then␊
# point to a directory where all project-related files (databases,␊
# keys, configurations) are kept, as these are automatically created␊
# on project creation by IDF.␊
#␊
# Example: 'mtn_repositories' is configured to be '/var/monotone/%s'␊
#␊
# - IDF tries to create /var/monotone/<projectname> as root directory␊
# - The database is placed in as /var/monotone/<projectname>/database.mtn␊
# - The server key is put into /var/monotone/<projectname>/keys and␊
# is named "<projectname>-server@<host>", where host is the host part␊
# of 'mtn_remote_url'␊
#␊
# therefor /var/monotone MUST be read/writable for the www user and all␊
# files which are created underknees MUST be read/writable by the user␊
# who is executing the usher instance! The best way to achieve this is with␊
# default (POSIX) ACLs on /var/monotone.␊
#␊
#␊
# You could also choose to setup usher by hand, i.e. with individual␊
# databases, in this case leave 'mtn_usher_conf' below commented out.␊
#␊
# Pro: - read and write access can be granted per project␊
# - no database locking issues␊
# - one public server running on the one well-known port␊
# Con: - harder to setup␊
#␊
# Usher can also be used to forward sync requests to remote servers,␊
# please consult its README file for more information.␊
#␊
# monotone also allows to use SSH as transport protocol, so if you do not plan␊
# to setup a netsync server as described above, then just enter a URI like␊
# 'ssh://my-host.biz/home/mtn/repositories/%s.mtn' in 'mtn_remote_url'.␊
#␊
␊
# The path to a specific database (local use) or a writable project␊
# directory (remote / usher use). %s is replaced with the project name␊
$cfg['mtn_repositories'] = '/home/mtn/repositories/%s.mtn';␊
␊
# The URL which is displayed as sync URL to the user and which is also␊
# used to connect to a remote usher␊
$cfg['mtn_remote_url'] = 'mtn://my-host.biz/%s';␊
#␊
␊
# Whether the particular database(s) are accessed locally (via automate stdio)␊
# or remotely (via automate remote_stdio). 'remote' is the default for␊
# netsync setups, while 'local' access should be choosed for ssh access.␊
#␊
# Note that you need to setup the hook 'get_remote_automate_permitted' for␊
# each remotely accessible database. A full HOWTO set this up is beyond this␊
# scope, please refer to the documentation of monotone and / or ask on the␊
# mailing list (monotone-users@nongnu.org) or IRC channel␊
# (irc.oftc.net/#monotone)␊
#␊
$cfg['mtn_db_access'] = 'remote';␊
#␊
# use with usher and the SyncMonotone plugin, while 'local' access should be␊
# choosed for manual setups and / or ssh access.␊
$cfg['mtn_db_access'] = 'local';␊
␊
# If true, each access to the database is authenticated with an auto-generated␊
# project key which is stored in the IDF project configuration␊
# ('mtn_client_key_*') and written out to $cfg['tmp_folder']/mtn-client-keys␊
# the remote monotone server instance. In this case no project-specific␊
# keys are generated and the server must be configured to allow at least␊
# anonymous read access to the main functions.␊
#␊
$cfg['mtn_remote_auth'] = true;␊
#␊
# If configured, this allows basic control of a running usher process␊
# via the forge administration. The variable must point to the full (writable)␊
#$cfg['mtn_remote_auth'] = true;␊
␊
# Needs to be configured for remote / usher usage.␊
# This allows basic control of a running usher process via the forge␊
# administration. The variable must point to the full (writable)␊
# path of the usher configuration file which gets updated when new projects␊
# are added␊
#␊
#$cfg['mtn_usher_conf'] = '/path/to/usher.conf';␊
␊
# Mercurial repositories path␊

Archive Download the corresponding diff file

Page rendered in 0.89312s using 13 queries.