Lsokolowski at 14:09, 20 May 2022

2022-05-20T14:09:47Z

New page

{{Cat|Subversion}}
{{SVN Links}}

<slideshow style="nobleprog" headingmark="⌘" incmark="…" scaled="false" font="Trebuchet MS" >
;title: Subversion for Administrators
;author: Bernard Szlachta (NobleProg Ltd), Lukasz Sokolowski
</slideshow>

== Subversion for Administrators ==
Subversion for Administrators Training Materials

{{Can I use your material}}

== Agenda ⌘==
* Repository Structure (trunk per project, many projects)
* Protocols (File Access, Standalone, Apache, SSH)
* Logging
* Lock administration
* Removing files from repository (svndumpfilter)
* Hooks
* Svnsync
* Upgrade/verify/dump/load/hotcopy
* Encrypted datatransfer

== Setting up Subversion ⌘==

* Choosing from connection-possiblilities (http://, https://, svn://, svn+ssh://)
* Choosing a repository structure (one Repository for all or one per project)?
* Choosing a permission structure (how many users and groups, who may read/write to which path/repository)?

== Files structure in repo ⌘==

* conf/
** This directory is a container for configuration files.
* db/
** This directory contains the data store for all of your versioned data.
* format
** This file describes the repository's internal organizational scheme.
* hooks/
** This directory contains hook script templates and hook scripts, if any have been installed.
* locks/
** Subversion uses this directory to house repository lock files, used for managing concurrent access to the repository.

== Subversion Commands ⌘==
* svn
* svnadmin
* svnlook
* svnserve
* svn2cl
* svndumpfilter
* svnrdump

=== svnadmin ⌘===
Tool for maintenance of repo

Removing dead transactions
svnadmin lstxns myrepos
svnadmin rmtxns myrepos `svnadmin lstxns myrepos`

Packing FSFS filesystems
svnadmin pack /var/svn/repos

=== svnlook ⌘===
Tool for diagnostic purposes
* for examining the various revisions and transactions (which are revisions in the making) in a repository
* do not change the repo
* typically used by the repository hooks for reporting the changes that
** are about to be committed
** or that were just committed

=== svnserve ⌘===

simple standalone service

=== svndumpfilter ⌘===

to easily modify streams of Subversion repository history data by acting as a path-based filter

=== svnrdump ⌘===

svnadmin dump and svnadmin load subcommands, rolled up into a separate program

== Repository Structure ⌘==
[[File:SvnArchitAdmin.png|800px]]

=== Repo Structure ⌘===

Module, Project, Library, etc...
* Repository per project
* Modules in root
* Modules in trunk
** Modules tagged separately
** Modules tagged together

==== Modules in root ⌘====
{|
|-
| [[File:SvnModRootAdmin.png|200px]] ||
* each module has to be checked out separately
* commits can not span multiple modules
* tags can not span multiple modules in single commit
* most common layout
|}

==== Modules in trunk ⌘====
{|
|-
| [[File:SvnModTrunkAdmin.png|200px]] ||
Modules tagged separately
* checkout of all modules is possible
* commits can span multiple modules
* tags can not span multiple modules in single commit (workaround by 3rd party tools)‏
|}

==== Modules in trunk con't ⌘====
{|
|-
| [[File:SvnModTrunkTogAdmin.png|200px]] ||
Modules tagged together
* checkout of all modules is possible
* commits can span multiple modules
* tags can span multiple modules in single commit
* maybe difficult to determine relevant modules of a tag
|}

== SVN Backend ⌘==
{| class="wikitable"
|-
! Header text !! FileSystem [„fsfs“] !! Berkeley DB[„bdb“]
|-
| Data integrity || stable || Difficult to deploy, but extremely reliable
|-
| Robustness || robust || fragile (“wedged”)
|-
| Usable from a read-only mount || yes || no
|-
| Platform-independent storage || yes || no
|-
| Usable over network filesystems || yes || no
|-
| Repository size|| smaller || larger
|-
| Scalability: number of revision trees || Depends on OS/FS (fixed in SVN 1.5) || No problems
|-
| Large commits || faster || slower
|-
| Directories with many files || faster || slower
|-
| Group permissions handling (unix) || no problems || Umask problem
|}

== Creating a repository ⌘==
you can create repositories in any empty directory:

>svnadmin create /path/to/repository
>svnadmin create --fs-type fsfs /var/svn/repos
>svnadmin create --fs-type bdb /var/svn/repos

== Protocol Comparison ⌘==
{| class="wikitable"
|-
! !! http:// https:// !! svn:// !! svn+ssh://
|-
| authentication|| all apache auth methods|| CRAM-MD5, SASL|| SSH
|-
| useraccount|| private userfiles or any other auth method|| private userfiles or SASL|| system accounts
|-
| authorization|| path-based authorization|| path-based authorization|| no path-based authorization
|-
| encryption|| optional SSL|| optional SASL|| always SSH
|-
| logging|| special configured logs|| no logging|| no logging
|-
| interoperation|| WebDAV clients|| svn clients only|| svn clients only
|-
| web access|| limited support|| no support|| no support
|-
| speed|| slower|| faster|| faster
|-
| complexity|| fair|| simple|| fair
|}

== Svnserve daemon ⌘==
Setting up Subversion as a standalone service just requires a simple command:
>svnserve –d –r [path to repository|repositories]

TEST:
>svn ls svn://localhost/[reponame]

== Windows Service ⌘==
>sc.exe create svn
binpath="c:\svn\bin\svnserve –-service –r [path to repo]"
displayname="Subversion Server"
depend= Tcpip
start=auto

== Svnserve authentication (1/4) ⌘==
3 files for authentication (in repo/config directory):
* svnserve.conf - general configuration file
* authz – stores path based authentication data
* passwd – stores user/password combinations

== Svnserve authentication (2/4) ⌘==
<pre>
svnserve.conf
[general]
anon-access = read
auth-access = write
password-db = passwd
authz-db = authz
realm = My First Repository

controls what anonym users can do (read, write or none)
controls what authenticated users can do (again, read, write or none)
location of the password database
location of the authz-db
authentication realm
</pre>

== Svnserve authentication(3/4) ⌘==
<pre>
authz
[aliases]
[groups]
devs = user1, user2
[/foo/bar]
username =
[repository:/baz/fuz]
@devs = rw

aliases can provide alternative names
groups of users
a path in repository. Next lines will be either aliases(&), groups (@) or users followed by either “r” for read, “rw” for read-write, or nothing for no access
repositoryname can be addressed with colon(:)
Same fileformat as for apache authzfile
</pre>

== Svnserve authentication(4/4) ⌘==
passwd
[users]
username = password
username2 = password

only a single user section

username followed by password (in clear text!)

== Exercise ⌘==
* Create repo in the /repo directory
* Create two directories harry and sally (use file protocol)
* Create two user sally/sally and harry/harry and grant access each user access to write their own directory
* Test it using svnserve –d and svn protocol

== Apache (single repository) ⌘==
<pre>
LoadModule dav_module modules/mod_dav.so
LoadModule dav_svn_module modules/mod_dav_svn.so
[...]
<Location /svn>
DAV svn
SVNPath path/to/repository
</Location>

dav module must be loaded before dav_svn module (2)
dav svn module translates http-dav requests into svn operations
location on server where repository will be served
activation of svn dav module
path to Subversion repository to be served
</pre>

== Exercise ⌘==
Make /repo repository available at http://localhost/repo

== apache (multi repos) ⌘==
<pre>
Multi repository(without path based authorization):

LoadModule dav_module modules/mod_dav.so
LoadModule dav_svn_module modules/mod_dav_svn.so
[...]
<Location /svn>
DAV svn
SVNParentPath path/to/repositories
</Location>

dav module must be loaded before dav_svn module (2)
dav svn module translates http-dav requests into svn operations
location on server where repository will be served
activation of svn dav module
path to folder with Subversion repositories to be served
</pre>

== apache authentication (1/3) ⌘==
<pre>
Basic authentication (without path based authorization):
<Location /svn>
DAV svn
SVNParentPath path/to/repositories
AuthType Basic
AuthName "Subversion repository"
AuthUserFile /path/to/http-auth-file
Require valid-user
</Location>
selection of authentication algorithm („Basic“ means password will not be encrypted)
the name of the authentication realm (will be shown on password entry)
file with user/passwords (use htpasswd to create file and add user/passwords)
only valid users can access this location
</pre>

== apache authentication (2/3) ⌘==
<pre>
Basic authentication, anonym read access (without path based authorization):
<Location /svn>
DAV svn
SVNParentPath path/to/repositories
AuthType Basic
AuthName "Subversion repository"
AuthUserFile /path/to/http-auth-file
<LimitExcept GET PROPFIND OPTIONS REPORT>
Require valid-user
</LimitExcept>
</Location>
makes exception for all http-DAV read operations, so anonymous user can read
</pre>

== apache authentication (3/3) ⌘==
<pre>
Path based authorization :
LoadModule authz_svn_module modules/mod_authz_svn.so
[...]
<Location /svn>
DAV svn
SVNParentPath path/to/repositories
AuthType Basic
AuthName "Subversion repository"
AuthUserFile /path/to/http-auth-file
Require valid-user
AuthzSVNAccessFile /path/to/access/file
</Location>
need to load authz_svn module to enable path based authentication
location of authz accessfile
</pre>

== Exercise ⌘==
Enforce the constraint that sally can access only /repo/sally folder and harry can access /repo/harry folder with apache

== Logging (apache) ⌘==
apache webserver can be configured to log any Subversion high level operation. Place this line outside of a location section:

CustomLog logs/svn_logfile "%t %u %{SVN-ACTION}e" env=SVN-ACTION

The %t and %u are standard apache logging constants meaning time and user.

Logging is only available on apache installations

== svn+ssh ⌘==
* No special configuration required
* There is NO PATH-BASED authorization
* Good idea to use groups and
** umask 0002
** chmod g+ws

== *Setting UP svn+ssh ⌘==
# Install open-ssh server (e.g. apt-get install ssh)
# Setup a user if they don’t exists
# Create a group (e.g. groupadd grepo)
# Add the user to the group
#* e.g. usermod -a -G grepo ubuntu
# Change permission and ownership on the repo folders
#* chmod g+ws /repo –R
#* chown :grepo /repo -R
# Change default umask (e.g. in /etc/profile) to umask 0002

== Exercise ⌘==
* Enable Logging as in the example show on the previous slide
* Test with commit and checkout commands

== Apache PAM authentication* ⌘==
<pre>
<VirtualHost *:80>
ServerName svn.nobleprog.net
<Location /repo>
AuthType Basic
AuthName “Sample Repository"
AuthPAM_Enabled On
AuthGROUP_Enabled on
Require group hitra
AuthBasicAuthoritative off
AuthUserFile /dev/null
Require valid-user
DAV svn
SVNPath /repo
</Location>
</VirtualHost>
</pre>

== lock administration ⌘==
As administrator you can always kill locks to all files:
To list locked files, you can use the lslocks command:

>svnadmin lslocks /path/to/repo [path/inside/repo]

To remove the locks use rmlocks

>svnadmin rmlocks /path/to/repo path/to/locked/file

== changing log messages ⌘==
Log Messages are not versioned, so changes to them will overwrite the old log message

Only administrators with file-system access can change log-message using this command:

>svnadmin setlog {path/to/repo} -r REV MSG-FILE --bypass-hooks

--bypass-hooks is important, as it will allow administrator to change revision without calling the hook »pre-revprop-change«

Caution: old data is lost!

You can allow client-side revision property changes by activating pre-revprop-change-hook

== Exercise ⌘==

Change the last revision log message to “I changed it”

== Dump and Load ⌘==
<pre>
svnadmin create /repo1
svnadmin dump /repo > /tmp/dump
svnadmin load /repo1 < /tmp/dump
</pre>

== Removing files from repository ⌘==
Subversion cannot remove files from repository, so if you really want to delete any trace of a file, 
you need to filter a dump and load it back to a new repository.

[[File:SvnDelFileAdmin.png|600px]]

== svndumpfilter problems ⌘==

svndumpfilter has problems with trailing slashes: 
If your dumpfile has trailing slashes your include/exclude argument 
must have trailing slashes also and vice versa.

svndumpfilter has problems with copies:
As a copy must show their origin in Subversion it could happen 
that just its origin was filtered by svndumpfilter.

This is a system immanent behavior of Subversion and cannot be changed.

However there are a few rewritten dumpfilters with some quirks fixed:

http://furius.ca/pubcode/pub/conf/bin/svndumpfilter3.html

== svndumpfilter Example ⌘==
* Create a new repository with 3 Dirs:
** project_A
** project_B
** project_C
* and in each directory a file:
** file_a.txt
** file_b.txt
** file_c.txt
* Export repository to dumpfile:
** svnadmin dump {path/to/repo} > {path/to/dumpfile}
* run svndumpfilter (special windows syntax!):
** svndumpfilter exclude project_A < c:\dump.txt > c:\project_a_ex.txt
** svndumpfilter include project_A < c:\dump.txt > c:\project_a.txt
* compare the three dumpfiles

== Exercise ⌘==

* Split the repository into 2
** /harry_repo containing file:///repo/harry folder
** /sally_repo containing rest of the repository

== Hooks ⌘==

Hooks play an important role in customizing 
workflows and adapting Subversion for special needs.

Hooks are triggered by special repository events 
and depending on the return value Subversion 
approves or cancels the action.

The hook script’s environment is empty so you need absolute paths and set all needed variables explicitly.

== Hooks con't ⌘==

There are 9 different types of hooks for 4 different events during commits:

commit hooks (start-commit, pre-commit, post-commit)
lock hooks (pre-lock, post-lock)
unlock hooks (pre-unlock, post-unlock)
revprop-change hook (pre-revprop-change, post-revprop-change)

Messages sent to stderr will be marshalled to the svn client.

On pre- or start- hooks, returning a non-zero value usually cancels the action.

== Commit Hooks ⌘==

The commit is the only event which has 3 hooks: 

start-commit(REPO_PATH, USER_NAME, CAPS) – from v 1.5 
This hook is called right before any data is transferred to the server. 
Useful for maintenance lock-down, or extra user-authentication. 

pre-commit(REPO_PATH, TXN_NAME) 
This hook is called right after data is transferred to the server, 
before the transaction is finished. 
Here you can check all different semantics by examinig the transaction.
Useful for enforcing workflows. 

post-commit(REPO_PATH, NEW_REVISION) 
This hook is called after transaction is complete, before acknowledge is sent back to client. 
Useful for sending notification mails. 

== Commit hook example ⌘==
<pre>
check for empty commit message:
REPOS="$1"
TXN="$2"

SVNLOOK=/usr/local/bin/svnlook
$SVNLOOK log -t "$TXN" "$REPOS" | \
grep "[a-zA-Z0-9]" > /dev/null || exit 1

exit 0
</pre>

== Commit hook example in Windows ⌘==
<pre>
"C:\Program Files\TortoiseSVN\bin\svnlook.exe" log -t %2 %1 | FindStr [a-zA-Z0-9]
IF %ERRORLEVEL% EQU 0 GOTO OK
echo "Commit Comments are Required" >&2
exit 1
:OK
exit 0
</pre>

== lock/unlock hooks ⌘==
<pre>
For customizing locking in Subversion you need lock/unlock hooks

lock hooks:
pre-lock(REPOS_PATH, PATH, USER, LOCK_COMMENT, STEAL[1:0])
Here you can define workflows (e.g. disallow stealing locks, allowing locks only for certain files)

post-lock(REPOS_PATH, USER) < locked path via stdin
Useful for notifications.

unlock hooks:
pre-unlock(REPOS_PATH, PATH, USER, TOKEN, BREAK:UNLOCK[1:0])
define workflow for releasing (breaking) locks, e.g. define administrator
who is allowed to break locks or notify previous lock-owner

post-unlock()
Useful for notifications.
</pre>

== revprop-change hooks ⌘==
We strongly advise against enabling revprop-changes, as backup and recovery will be much more complex.
<pre>
revprop-change hooks:
pre-revprop-change(REPOS_PATH, REV, USER, PROPNAME, ACTION)
used for semantics to control who can change the revision properties. On stdin is the old property value

post-revprop-change(REPOS_PATH, REV, USER, PROPNAME, ACTION)
used for notification and backup purposes
</pre>
== revprop-change hook in windows ⌘==
<pre>
@ECHO OFF
:: Set all parameters. Even though most are not used, in case you want to add
:: changes that allow, for example, editing of the author or addition of log messages.
set repository=%1
set revision=%2
set userName=%3
set propertyName=%4
set action=%5

:: Only allow the log message to be changed, but not author, etc.
if /I not "%propertyName%" == "svn:log" goto ERROR_PROPNAME

:: Only allow modification of a log message, not addition or deletion.
if /I not "%action%" == "M" goto ERROR_ACTION

:: Make sure that the new svn:log message is not empty.
set bIsEmpty=true
for /f "tokens=*" %%g in ('find /V ""') do (
set bIsEmpty=false
)
if "%bIsEmpty%" == "true" goto ERROR_EMPTY

goto :eof

:ERROR_EMPTY
echo Empty svn:log messages are not allowed. >&2
goto ERROR_EXIT

:ERROR_PROPNAME
echo Only changes to svn:log messages are allowed. >&2
goto ERROR_EXIT

:ERROR_ACTION
echo Only modifications to svn:log revision properties are allowed. >&2
goto ERROR_EXIT

:ERROR_EXIT
exit /b 1
</pre>

== Exercises ⌘==
* Copy pre-commit.tmpl file to pre-commit and test whether you can commit something without specifying a message
* Create 'pre-revprop-change' hook and allow only to modify 'svn:log' property

== svnsync ⌘==
[[File:SvnSyncAdmin.png|600px]]

== svnsync con't ⌘==

create mirror repository
>svnadmin create /path/to/mirror

create a pre-revprop-change hookscript in mirror repository
>echo exit 0 > /path/to/mirror/hooks/pre-revprop-change
>chmod +x /path/to/mirror/hooks/pre-revprop-change

initialize mirror repository
>svnsync init file:///path/to/mirror file:///path/to/repo

synchronize repositories
>svnsync sync file:///path/to/mirror

== Backups – hot copy ⌘==

copy a repository (even on a running server!) with the hotcopy command:
>svnadmin hotcopy /path/to/repo /path/to/new/repo

Note that hooks will be copied, too.

[[File:SvnHotAdmin.png|400px]]

== Hot Copies ⌘==
Hotcopy copies the repository along with all other
file configurations, permissions, hooks and logs.

* Hotcopy works on a running system regardless of running transactions
* Hotcopy creates an exact copy of a new repository which can be used as a backup.
* Hotcopy uses as much diskspace as the original repository.
* Hotcopies are faster to restore than dumpfiles.

== Repository Integrity ⌘==
You should check the integrity of your repositories on a regular basis

Use the verify command:

>svnadmin verify /path/to/repo

== Dumps ⌘==

* Dump is the secure way of exporting a repository with all history and versioning information.
* Human readable fileformat.
* Binary files exist inline so you should not edit the dump-file with your normal texteditor.
* Files are quite large but compress well.
* Through the --incremental switch dumps can be saved from revision to revision.
* Restoring a broken repository from a dump might take a long time.

== Upgrading Repository ⌘==

You can use a dump/load cycle to upgrade the repository format:
>svnadmin create /new/repo
>svnadmin dump /path/to/repo | svnadmin load /new/repo
[[File:SvnUpgrAdmin.png|400px]]

== Upgrading con't ⌘==

Since svn 1.5 you can use the upgrade command to upgrade the repository format
>svnadmin upgrade /path/to/repo

[[File:SvnUpgr2Admin.png|400px]]

This option is much faster, however, you will not get all server-side repository features.

== GUI ⌘==
* Plugins
** subversive (eclipse plugin)
** netbeans, etc
* tortoise
* VisualSVN

Subversion for Administrators - Revision history

Lsokolowski at 14:09, 20 May 2022