Subversion repository mirror

Creating a mirror of your Subversion repository means you have an up to date copy of your repository in a safe place. This guide shows how to set up a mirror and configure it for synchronisation. The instructions are for Windows.

Setting up the mirror uses svnsync which can be very slow over the network. It’s better to take a dump of the repository and load that into the mirror before starting the sync.

Take a backup

On the central repository:
$ cd c:\temp
$ svnadmin dump c:\subversion\my_repository --incremental -q > my-repository.dump
Create a 7zip archive of the dump file and delete the dump.
Copy the archived dump file to the repository mirror machine.

Create the mirror

On the mirror’s machine:
$ mkdir c:\repos && cd c:\repos
$ svnadmin create my-mirror

Load the dump file.
$ svnadmin load my-mirror -q < c:\temp\my-repository.dump

Serve up.
$ svnserve -d -r c:\repos\my-mirror
$ svn info svn://localhost/

Allow write access

conf/svnserve.conf
anon-access = none
auth-access = write
password-db = passwd.txt
authz-db = authz.txt

conf/passwd.txt
[users]
repobot = Robot1

conf/authz.txt
[/]
repobot = rw

Note: It can be easier to set up with anon-access = write and once it’s all working then upgrade the security.

Set up a pre-revision hook

[http://www.cardinalpath.com/how-to-use-svnsync-to-create-a-mirror-backup-of-your-subversion-repository/ > Step 3.]
$ cp C:\repos\my-mirror\hooks\pre-revprop-change.tmpl pre-revprop-change.bat
@ECHO OFF
set repository=%1
set revision=%2
set userName=%3
set propertyName=%4
set action=%5

:: Allow editing of revision properties for user.
if "%userName%" == "repobot" goto :eof else goto :ERROR_USER

:ERROR_USER
echo Only admin user may change revision properties. >&2
goto ERROR_EXIT

:ERROR_EXIT
exit 1

Set up properties to allow svnsync with the pre-populated mirror

$ svn info https://repo-box:8443/svn/my_repository

Use the UUID of the remote:
$ svn propset --revprop -r0 svn:sync-from-uuid 5aab9995-1597-de4e-ac02-ede68baf940a svn://localhost/

Use the last merged revision number:
$ svn propset --revprop -r0 svn:sync-last-merged-rev 14175 svn://localhost/

Set to sync from url:
$ svn propset --revprop -r0 svn:sync-from-url https://repo-box:8443/svn/my_repository svn://localhost/

Initial Sync

$ svnsync synchronize svn://localhost/

Auto-sync

repo-box:
hooks/post-commit.bat
svnsync --non-interactive --steal-lock --sync-username repobot --sync-password Robot1 --source-username repobot --source-password Robot1 sync svn://mirror-box/ svn://repo-box/

$ sc create SvnServe binpath= "\"%programfiles%\CollabNet\Subversion Client\svnserve.exe\" --service -r c:\subversion\my_repository" depend= Tcpip start= auto
$ net start svnserve

Create Windows Service for svnserve

sc delete SvnServe
sc create SvnServe binPath= "%programfiles%\TortoiseSVN\bin\svnserve.exe --service -r c:\repos\my-mirror" depend= Tcpip start= auto
net start SvnServe

Mercurial with a Subversion central repository

This How-To is for developers wishing to use Mercurial (Hg) on their Windows development boxes when the source code is in a central Subversion (SVN) repository.

Install TortoiseHg.

In a Windows development environment the easiest way to get started with Hg is to install TortoiseHg. The Windows Explorer extension not only makes it easy to work with files by using icon overlays and context menus but also packages up Python, the SVN bindings and a large number of Hg extensions that will prove very useful.

If you are behind a proxy you must first configure Hg with the proxy settings. Use the TortoiseHg context menu and Global Settings > Proxy.

Working with SVN.

Given that you are working against a central SVN repository then Hg has a number of extensions that can talk to the SVN repo. I use HgSubversion. To install this extension you need to clone the hgsubversion repo:

hg clone http://bitbucket.org/durin42/hgsubversion c:\hgsvn

Use the TortoiseHg context menu and Global Settings > Edit File.

[extensions]
hgsubversion = c:\hgsvn

Start with a single project. It’s better to clone the whole repo, the trunk, branches and tags, rather than just a branch. This way you only need to do the clone once – clones can take a while. Create a project folder, i.e. c:\development\local\myproject, change to it and then clone the SVN repo:

hg clone sv://subversion/myproject/


Ignore.

It’s best to set up the ignore list for Hg now. Save this as .hgignore and copy it to the root of your local Hg clone (next to .hg).

# Mercurial .hgignore file template by Abdullin.com
# For syntax see: http://linux.die.net/man/5/hgignore
# Source: http://bitbucket.org/abdullin/snippets/

# Visual Studio
glob:*.user
glob:*.suo
glob:*.cache
glob:_ReSharper.*/
relre:/obj/
relre:/bin/
relre:/Bin/

# Subversion
glob:.svn/
glob:_svn/

# Build structure
relre:^Build/

# Misc
glob:Thumbs.db
glob:*.bak
glob:*.log


Local Development.

If you use local branches and merge them in your local Hg repo you won’t be able to push changes to SVN.

If you commit often then you will end up with lots of changesets to push to SVN that will make it harder to see what the intention of your update was.

A good way to solve both problems is to submit patches to the central repo. Hg has an extension for patch management called Mercurial Queues (MQ).

Mercurial Queues.

Mercurial Queues (MQ) provides patch management integrated with Hg. This is incredibly useful for packaging up a lot of small changes that you record as you work locally into a single submission to the central repo. You can also work on a number of patches concurrently.

Enable the MQ extension in Hg, TortoiseHg > Global Settings > Extensions and check ‘mq’. In the mercurial.ini set diffs to use the git format.

[diff]
git = True

In the Hg Repository Explorer you can create a new patch from the latest revision. This means you will need to commit something you want to go into a patch before you can create the patch. The other way is to use the command line.

hg qnew mylatestpatch

qnew will include the latest changes found in the working directory.

qfold

One way to work with plenty of local commits but to have a single, tidy submission to the central repo is to use MQ’s qfold to create a patch.

Once you have some changesets you wish to publish you convert them into patches. In Hg Repository Explorer use the context menu on the latest changeset first: Mercurial Queues > Import Revision to MQ. Repeat for each of the changesets. Next, unapply the latest patches until just the earliest is still applied. Now at the command line use qfold. When you refresh Hg Repository Explorer you will see a single patch. Use the patch context menu’s ‘Finish Applied’ to change the patch into a single changeset you can push to the central repo.

For example, you have three changesets: 16; 17 and 18. You then create three patches: 16.diff; 17.diff and 18.diff. You unapply 17.diff and 18.diff then go to the command line:

hg qfold 17
hg qfold 18

Now you have a single patch, 16.diff, that contains all the work commited in the original three changesets.

A working copy for the weekend

Sometimes it’s good to be able to review some code out of working hours when you suddenly have some inspiration. Using nncron, svn export, 7z and FTP (TLS) I’ve been able to automate this quite easily.

In cron.tab:
# at 5pm export a working copy, archive it and transfer the archive by FTP over TLS.
0 17 * * 1-5 CMD /K "C:\Documents and Settings\Placeholder\My Documents\scripts\svn_export.bat"

In the batch file:
set project=myProject
REM Export the working copy.
"C:\Program Files\Subversion\bin\svn.exe" export --ignore-externals C:\svn\%project% C:\svn\export\%project%
REM Create an archive of the files.
"C:\Program Files\7-Zip\7z.exe" u "C:\svn\export\%project%.7z" "C:\svn\export\%project%\*"
REM Move the archive to another location.
"C:\Program Files\BlueZone FTP\Bzftpf.exe" /Fdummy.zft /Z0 /L"MPUT *.7z" /L"EXIT"


I used the BlueZone Secure FTP Client as it supports FTP over TLS (I don’t want my login being broadcast) and it was scriptable. I use 7-Zip as the 7z archive is smaller than a ZIP file (over 40% smaller). I had to install Subversion as although TortoiseSVN is scriptable its ‘export’ function wasn’t exactly what I wanted (my current working copy without unversioned files).

Enabling revision of props

It is useful to be able to alter the log messages of SVN revisions but this isn’t possible with a default installation.

By default Subversion disallows the revision of properties on folders. These props aren’t under source control so it makes sense as you could end up with no-blame, uncontrolled alterations. In the ‘hooks’ folder of the repository there are a bunch of templates. Take the pre-revprop-change.tmpl and change the extension to .bat. If it was just left empty then the exit code would allow Subversion to make prop changes to revision. The following DOS batch file, however, limits those changes to log messages only.

@ECHO OFF
REM pre-revprop-change.bat

SET REPOS=%1
SET REV=%2
SET USER=%3
SET PROPNAME=%4

SET SVNLOG=svn:log

IF %PROPNAME% == %SVNLOG% GOTO :LOGEXIT
EXIT 1
:LOGEXIT
EXIT 0


Properties for Subversion folders

More tricky than it appears under Windows, especially if you’re using a network drive path because it’s a guerilla dev installation. You can’t set the props through the command line, it appears, but you can through Tortoise. The correct field for svn:externals was:

tools file:///J:/repo/tools

Setting this prop on the directory that will be the parent of the ‘tools’ directory worked just fine. Now we only need one tools directory in the repo and the export of the source will be much cleaner.

svnserve

The following command line created a useful SVN server:

sc create svnserve binpath= "\"E:\Program Files\Subversion\bin\svnserve.exe\" --service --root F:\SVN\Repository" start= auto depend= Tcpip type= own type= interact displayname= "Subversion Repository"

It can be accessed with:

svn://localhost/

The reason for setting this up was the very poor response time of using a network share over Microsoft Loopback Adapter in a Virtual PC desktop environment [mentioned here].

VPC, ICS, ZA

This all started with Subversion throwing a wobbly and not letting me see my repository in my Virtual PC (VPC) shared folder. It reported the file system as RAW even though it would work fine in the host and if I looked at a hotcopy of the repository it was fine. I decided, after the problem “went away”, that I should move away from shared folders (they’re terribly slow).

To do this required installing the Microsoft Loopback Adapter on the host and installing Internet Connection Sharing (ICS) for the new network adapter on the main NIC (currently the WiFi card although I should really bridge that with the ethernet network card). I could ping from host to guest and vice versa but that was all. Finally discovered it was ZoneAlarm (ZA) on the host being an arse. Even after you’ve added the new network as a trusted zone you have to lower the Internet Zone Security to medium. There is a setting in the, almost invisible, Advanced tab on the Firewall config page that is supposed to allow ICS but that doesn’t make any difference.

Anyway, now I have a mapped drive in my guest to my host data drive and it’s a lot faster than before. I’ll have to see if changing the Shared networking (NAT) network adaptor in the VPC settings makes any difference to the Local connection for my virtual network server setup.

Script for Subversion hotcopy

REM Backup the Subversion repositories.

REM 'repo' is the working copy parent path.
SET repo="J:\repository"

REM 'backup' is the path to the backup folder.
SET backup="C:\SVN\Backup"

REM 'old' is a backup of the last backup in case the script fails.
SET old="C:\SVN\Old"

REM 'hotcopy' needs empty directories to copy into.
RMDIR /s /q %old%
REN %backup% Old
MKDIR %backup%

svnadmin hotcopy %repo% %backup% --clean-logs