Subversion for Administrators

title: Subversion for Administrators
author: Bernard Szlachta (NobleProg Ltd), Lukasz Sokolowski

Subversion for Administrators

Subversion for Administrators Training Materials

Copyright Notice

This publication is protected by copyright, and permission must be obtained from the publisher prior to any prohibited reproduction, storage in a retrieval system, or transmission in any form or by any means, electronic, mechanical, photocopying, recording, or likewise.

Agenda ⌘

Repository Structure (trunk per project, many projects)
Protocols (File Access, Standalone, Apache, SSH)
Logging
Lock administration
Removing files from repository (svndumpfilter)
Hooks
Svnsync
Upgrade/verify/dump/load/hotcopy
Encrypted datatransfer

Setting up Subversion ⌘

Choosing from connection-possiblilities (http://, https://, svn://, svn+ssh://)
Choosing a repository structure (one Repository for all or one per project)?
Choosing a permission structure (how many users and groups, who may read/write to which path/repository)?

Files structure in repo ⌘

conf/
- This directory is a container for configuration files.
db/
- This directory contains the data store for all of your versioned data.
format
- This file describes the repository's internal organizational scheme.
hooks/
- This directory contains hook script templates and hook scripts, if any have been installed.
locks/
- Subversion uses this directory to house repository lock files,
  used for managing concurrent access to the repository.

Subversion Commands ⌘

svn
svnadmin
svnlook
svnserve
svn2cl
svndumpfilter
svnrdump

svnadmin ⌘

Tool for maintenance of repo

Removing dead transactions

svnadmin lstxns myrepos
svnadmin rmtxns myrepos `svnadmin lstxns myrepos`

Packing FSFS filesystems

svnadmin pack /var/svn/repos

svnlook ⌘

Tool for diagnostic purposes

for examining the various revisions and transactions (which are revisions in the making) in a repository
do not change the repo
typically used by the repository hooks for reporting the changes that
- are about to be committed
- or that were just committed

svnserve ⌘

simple standalone service

svndumpfilter ⌘

to easily modify streams of Subversion repository history data by acting as a path-based filter

svnrdump ⌘

svnadmin dump and svnadmin load subcommands, rolled up into a separate program

Repository Structure ⌘

Repo Structure ⌘

Module, Project, Library, etc...

Repository per project
Modules in root
Modules in trunk
- Modules tagged separately
- Modules tagged together

Modules in root ⌘

each module has to be checked out separately
commits can not span multiple modules
tags can not span multiple modules in single commit
most common layout

Modules in trunk ⌘

Modules tagged separately

checkout of all modules is possible
commits can span multiple modules
tags can not span multiple modules in single commit (workaround by 3rd party tools)‏

Modules in trunk con't ⌘

Modules tagged together

checkout of all modules is possible
commits can span multiple modules
tags can span multiple modules in single commit
maybe difficult to determine relevant modules of a tag

SVN Backend ⌘

Header text	FileSystem [„fsfs“]	Berkeley DB[„bdb“]
Data integrity	stable	Difficult to deploy, but extremely reliable
Robustness	robust	fragile (“wedged”)
Usable from a read-only mount	yes	no
Platform-independent storage	yes	no
Usable over network filesystems	yes	no
Repository size	smaller	larger
Scalability: number of revision trees	Depends on OS/FS (fixed in SVN 1.5)	No problems
Large commits	faster	slower
Directories with many files	faster	slower
Group permissions handling (unix)	no problems	Umask problem

Creating a repository ⌘

you can create repositories in any empty directory:

>svnadmin create /path/to/repository
>svnadmin create --fs-type fsfs /var/svn/repos
>svnadmin create --fs-type bdb /var/svn/repos

Protocol Comparison ⌘

	http:// https://	svn://	svn+ssh://
authentication	all apache auth methods	CRAM-MD5, SASL	SSH
useraccount	private userfiles or any other auth method	private userfiles or SASL	system accounts
authorization	path-based authorization	path-based authorization	no path-based authorization
encryption	optional SSL	optional SASL	always SSH
logging	special configured logs	no logging	no logging
interoperation	WebDAV clients	svn clients only	svn clients only
web access	limited support	no support	no support
speed	slower	faster	faster
complexity	fair	simple	fair

Svnserve daemon ⌘

Setting up Subversion as a standalone service just requires a simple command:

>svnserve –d –r [path to repository|repositories]

TEST:

>svn ls svn://localhost/[reponame]

Windows Service ⌘

>sc.exe create svn 
 binpath="c:\svn\bin\svnserve –-service –r [path to repo]"
 displayname="Subversion Server"
 depend= Tcpip
 start=auto

Svnserve authentication (1/4) ⌘

3 files for authentication (in repo/config directory):

svnserve.conf - general configuration file
authz – stores path based authentication data
passwd – stores user/password combinations

Svnserve authentication (2/4) ⌘

svnserve.conf
[general]
anon-access = read
auth-access = write
password-db = passwd
authz-db = authz
realm = My First Repository

controls what anonym users can do (read, write or none)
controls what authenticated users can do (again, read, write or none)
location of the password database
location of the authz-db
authentication realm

Svnserve authentication(3/4) ⌘

authz
[aliases]
[groups]
devs = user1, user2
[/foo/bar]
username =
[repository:/baz/fuz]
@devs = rw

aliases can provide alternative names 
groups of users 
a path in repository. Next lines will be either aliases(&), groups (@) or users followed by either “r” for read, “rw” for read-write, or nothing for no access
repositoryname can be addressed with colon(:)
Same fileformat as for apache authzfile

Svnserve authentication(4/4) ⌘

passwd

[users]
username = password
username2 = password

only a single user section

username followed by password (in clear text!)

Exercise ⌘

Create repo in the /repo directory
Create two directories harry and sally (use file protocol)
Create two user sally/sally and harry/harry and grant access each user access to write their own directory
Test it using svnserve –d and svn protocol

Apache (single repository) ⌘

LoadModule dav_module      modules/mod_dav.so
LoadModule dav_svn_module  modules/mod_dav_svn.so
[...]
<Location /svn>
  DAV svn
  SVNPath path/to/repository
</Location>

dav module must be loaded before dav_svn module (2)
dav svn module translates http-dav requests into svn operations
location on server where repository will be served
activation of svn dav module
path to Subversion repository to be served

Exercise ⌘

Make /repo repository available at http://localhost/repo

apache (multi repos) ⌘

Multi repository(without path based authorization):

LoadModule dav_module      modules/mod_dav.so
LoadModule dav_svn_module  modules/mod_dav_svn.so
[...]
<Location /svn>
  DAV svn
  SVNParentPath path/to/repositories
</Location>

dav module must be loaded before dav_svn module (2)
dav svn module translates http-dav requests into svn operations
location on server where repository will be served
activation of svn dav module
path to folder with Subversion repositories  to be served

apache authentication (1/3) ⌘

Basic authentication (without path based authorization):
<Location /svn>
  DAV svn
  SVNParentPath path/to/repositories
  AuthType Basic
  AuthName "Subversion repository"
  AuthUserFile /path/to/http-auth-file
  Require valid-user
</Location>
selection of authentication algorithm („Basic“ means password will not be encrypted)
the name of the authentication realm (will be shown on password entry)
file with user/passwords (use htpasswd to create file and add user/passwords)
only valid users can access this location

apache authentication (2/3) ⌘

Basic authentication, anonym read access (without path based authorization):
<Location /svn>
  DAV svn
  SVNParentPath path/to/repositories
  AuthType Basic
  AuthName "Subversion repository"
  AuthUserFile /path/to/http-auth-file
<LimitExcept GET PROPFIND OPTIONS REPORT>
    Require valid-user
  </LimitExcept>
</Location>
makes exception for all http-DAV read operations, so anonymous user can read

apache authentication (3/3) ⌘

Path based authorization :
LoadModule authz_svn_module   modules/mod_authz_svn.so
[...]
<Location /svn>
  DAV svn
  SVNParentPath path/to/repositories
  AuthType Basic
  AuthName "Subversion repository"
  AuthUserFile /path/to/http-auth-file
  Require valid-user
  AuthzSVNAccessFile /path/to/access/file 
</Location>
need to load authz_svn module to enable path based authentication
location of authz accessfile

Exercise ⌘

Enforce the constraint that sally can access only /repo/sally folder and harry can access /repo/harry folder with apache

Logging (apache) ⌘

apache webserver can be configured to log any Subversion high level operation. Place this line outside of a location section:

CustomLog logs/svn_logfile "%t %u %{SVN-ACTION}e" env=SVN-ACTION

The %t and %u are standard apache logging constants meaning time and user.

Logging is only available on apache installations

svn+ssh ⌘

No special configuration required
There is NO PATH-BASED authorization
Good idea to use groups and
- umask 0002
- chmod g+ws

*Setting UP svn+ssh ⌘

Install open-ssh server (e.g. apt-get install ssh)
Setup a user if they don’t exists
Create a group (e.g. groupadd grepo)
Add the user to the group
- e.g. usermod -a -G grepo ubuntu
Change permission and ownership on the repo folders
- chmod g+ws /repo –R
- chown :grepo /repo -R
Change default umask (e.g. in /etc/profile) to umask 0002

Exercise ⌘

Enable Logging as in the example show on the previous slide
Test with commit and checkout commands

Apache PAM authentication* ⌘

<VirtualHost *:80>
        ServerName svn.nobleprog.net
<Location /repo>
        AuthType Basic
        AuthName “Sample Repository"
        AuthPAM_Enabled On
        AuthGROUP_Enabled on
        Require group hitra
        AuthBasicAuthoritative off
        AuthUserFile /dev/null
        Require valid-user
        DAV svn
        SVNPath /repo
    </Location>
</VirtualHost>

lock administration ⌘

As administrator you can always kill locks to all files: To list locked files, you can use the lslocks command:

>svnadmin lslocks /path/to/repo [path/inside/repo]

To remove the locks use rmlocks

>svnadmin rmlocks /path/to/repo path/to/locked/file

changing log messages ⌘

Log Messages are not versioned, so changes to them will overwrite the old log message

Only administrators with file-system access can change log-message using this command:

>svnadmin setlog {path/to/repo} -r REV MSG-FILE --bypass-hooks

--bypass-hooks is important, as it will allow administrator to change revision without calling the hook »pre-revprop-change«

Caution: old data is lost!

You can allow client-side revision property changes by activating pre-revprop-change-hook

Exercise ⌘

Change the last revision log message to “I changed it”

Dump and Load ⌘

svnadmin create /repo1
svnadmin dump /repo > /tmp/dump
svnadmin load /repo1 < /tmp/dump

Removing files from repository ⌘

Subversion cannot remove files from repository, so if you really want to delete any trace of a file,
you need to filter a dump and load it back to a new repository.

svndumpfilter problems ⌘

svndumpfilter has problems with trailing slashes:
If your dumpfile has trailing slashes your include/exclude argument
must have trailing slashes also and vice versa.

svndumpfilter has problems with copies: As a copy must show their origin in Subversion it could happen
that just its origin was filtered by svndumpfilter.

This is a system immanent behavior of Subversion and cannot be changed.

However there are a few rewritten dumpfilters with some quirks fixed:

http://furius.ca/pubcode/pub/conf/bin/svndumpfilter3.html

svndumpfilter Example ⌘

Create a new repository with 3 Dirs:
- project_A
- project_B
- project_C
and in each directory a file:
- file_a.txt
- file_b.txt
- file_c.txt
Export repository to dumpfile:
- svnadmin dump {path/to/repo} > {path/to/dumpfile}
run svndumpfilter (special windows syntax!):
- svndumpfilter exclude project_A < c:\dump.txt > c:\project_a_ex.txt
- svndumpfilter include project_A < c:\dump.txt > c:\project_a.txt
compare the three dumpfiles

Exercise ⌘

Split the repository into 2
- /harry_repo containing file:///repo/harry folder
- /sally_repo containing rest of the repository

Hooks ⌘

Hooks play an important role in customizing
workflows and adapting Subversion for special needs.

Hooks are triggered by special repository events
and depending on the return value Subversion
approves or cancels the action.

The hook script’s environment is empty so you need absolute paths and set all needed variables explicitly.

Hooks con't ⌘

There are 9 different types of hooks for 4 different events during commits:

commit hooks 		(start-commit, pre-commit, post-commit)
lock hooks 		(pre-lock, post-lock)
unlock hooks 		(pre-unlock, post-unlock)
revprop-change hook 	(pre-revprop-change, post-revprop-change)

Messages sent to stderr will be marshalled to the svn client.

On pre- or start- hooks, returning a non-zero value usually cancels the action.

Commit Hooks ⌘

The commit is the only event which has 3 hooks:

start-commit(REPO_PATH, USER_NAME, CAPS) – from v 1.5
This hook is called right before any data is transferred to the server.
Useful for maintenance lock-down, or extra user-authentication.

pre-commit(REPO_PATH, TXN_NAME)
This hook is called right after data is transferred to the server,
before the transaction is finished.
Here you can check all different semantics by examinig the transaction. Useful for enforcing workflows.

post-commit(REPO_PATH, NEW_REVISION)
This hook is called after transaction is complete, before acknowledge is sent back to client.
Useful for sending notification mails.

Commit hook example ⌘

check for empty commit message:
REPOS="$1"
TXN="$2"

SVNLOOK=/usr/local/bin/svnlook
$SVNLOOK log -t "$TXN" "$REPOS" | \
   grep "[a-zA-Z0-9]" > /dev/null || exit 1

exit 0

Commit hook example in Windows ⌘

"C:\Program Files\TortoiseSVN\bin\svnlook.exe" log -t %2 %1 | FindStr [a-zA-Z0-9]
IF %ERRORLEVEL% EQU 0 GOTO OK
echo "Commit Comments are Required" >&2
exit 1
:OK
exit 0

lock/unlock hooks ⌘

For customizing locking in Subversion you need lock/unlock hooks

lock hooks:
pre-lock(REPOS_PATH, PATH, USER, LOCK_COMMENT, STEAL[1:0])
Here you can define workflows (e.g. disallow stealing locks, allowing locks only for certain files)

post-lock(REPOS_PATH, USER) < locked path via stdin
Useful for notifications.

unlock hooks:		
pre-unlock(REPOS_PATH, PATH, USER, TOKEN, BREAK:UNLOCK[1:0])
define workflow for releasing (breaking) locks, e.g. define administrator 
who is allowed to break locks or notify previous lock-owner

post-unlock()
Useful for notifications.

revprop-change hooks ⌘

We strongly advise against enabling revprop-changes, as backup and recovery will be much more complex.

revprop-change hooks:
pre-revprop-change(REPOS_PATH, REV, USER, PROPNAME, ACTION) 
used for semantics to control who can change the revision properties. On stdin is the old property value

post-revprop-change(REPOS_PATH, REV, USER, PROPNAME, ACTION)
used for notification and backup purposes

revprop-change hook in windows ⌘

@ECHO OFF
:: Set all parameters. Even though most are not used, in case you want to add
:: changes that allow, for example, editing of the author or addition of log messages.
set repository=%1
set revision=%2
set userName=%3
set propertyName=%4
set action=%5

:: Only allow the log message to be changed, but not author, etc.
if /I not "%propertyName%" == "svn:log" goto ERROR_PROPNAME

:: Only allow modification of a log message, not addition or deletion.
if /I not "%action%" == "M" goto ERROR_ACTION

:: Make sure that the new svn:log message is not empty.
set bIsEmpty=true
for /f "tokens=*" %%g in ('find /V ""') do (
set bIsEmpty=false
)
if "%bIsEmpty%" == "true" goto ERROR_EMPTY

goto :eof

:ERROR_EMPTY
echo Empty svn:log messages are not allowed. >&2
goto ERROR_EXIT

:ERROR_PROPNAME
echo Only changes to svn:log messages are allowed. >&2
goto ERROR_EXIT

:ERROR_ACTION
echo Only modifications to svn:log revision properties are allowed. >&2
goto ERROR_EXIT

:ERROR_EXIT
exit /b 1

Exercises ⌘

Copy pre-commit.tmpl file to pre-commit and test whether you can commit something without specifying a message
Create 'pre-revprop-change' hook and allow only to modify 'svn:log' property

svnsync ⌘

svnsync con't ⌘

create mirror repository

>svnadmin create /path/to/mirror

create a pre-revprop-change hookscript in mirror repository

>echo exit 0 > /path/to/mirror/hooks/pre-revprop-change
>chmod +x /path/to/mirror/hooks/pre-revprop-change

initialize mirror repository

>svnsync init file:///path/to/mirror file:///path/to/repo

synchronize repositories

>svnsync sync file:///path/to/mirror

Backups – hot copy ⌘

copy a repository (even on a running server!) with the hotcopy command:

>svnadmin hotcopy /path/to/repo /path/to/new/repo

Note that hooks will be copied, too.

Hot Copies ⌘

Hotcopy copies the repository along with all other file configurations, permissions, hooks and logs.

Hotcopy works on a running system regardless of running transactions
Hotcopy creates an exact copy of a new repository which can be used as a backup.
Hotcopy uses as much diskspace as the original repository.
Hotcopies are faster to restore than dumpfiles.

Repository Integrity ⌘

You should check the integrity of your repositories on a regular basis

Use the verify command:

>svnadmin verify /path/to/repo

Dumps ⌘

Dump is the secure way of exporting a repository with all history and versioning information.
Human readable fileformat.
Binary files exist inline so you should not edit the dump-file with your normal texteditor.
Files are quite large but compress well.
Through the --incremental switch dumps can be saved from revision to revision.
Restoring a broken repository from a dump might take a long time.

Upgrading Repository ⌘

You can use a dump/load cycle to upgrade the repository format:

>svnadmin create /new/repo
>svnadmin dump /path/to/repo | svnadmin load /new/repo

Upgrading con't ⌘

Since svn 1.5 you can use the upgrade command to upgrade the repository format

>svnadmin upgrade /path/to/repo

This option is much faster, however, you will not get all server-side repository features.

GUI ⌘

Plugins
- subversive (eclipse plugin)
- netbeans, etc
tortoise
VisualSVN

Subversion for Administrators