Get Free Printed Copies Of The Book - Version Control By Example

Eric Sink the founder of SourceGear - a company that has developed a slew of products revolving around Version Control has published his book named - "Version Control by Example".

And guess what, he is offering a printed copy of the book for free to whoever asks for it.
Read more »

Filtering Subversion Commits Using a Post Commit Hook

At the end of a successful commit phase Subversion invokes a post commit hook, if it exists. The post commit hook is an executable file that must be named $SVNREPO/hooks/post-commit. The post commit hook is passed two parameters:

1. The repository affected by the commit operation.
2. The committed revision number.

With these parameters you can find out the changes that affected the repository using the svnlook command:

$ svnlook changed -r [revision] [repository]

If your post-commit hook is a shell script, you can just use:

[...snip...]

svnlook changed -r "$2" "$1"

[...snip...]

Unless you can control which the $PATH environment variable will be at the time of the hook execution, be sure to use full command paths in your scripts to avoid path related errors.

Example

If you want to filter out a file name, for example web.xml, from svnlook output, you can use the following syntax (please note that the following uses Solaris-specific commands):

MODIFICATIONS=$(/opt/csw/bin/svnlook changed -r "$2" "$REPOS")

for i in "$MODIFICATIONS" ; do

  echo $i | /usr/xpg4/bin/grep -q "web.xml$"

  if [ $? == 0 ] ; then

    # The $CHANGES variable of this example will contain

    # the list of the To: addresses for the current email.

    echo $i | mailx -s "Web Config files have been modified" $CHANGES

  fi

done

Solaris Specific Syntax

The -q option for the command grep is supported by the XPG4 version of the grep command, which is bundled with Solaris and installed by default in the /usr/xpg4/bin directory.

Note about sending an email on Solaris

To send an email on an UNIX system without worrying about the specific infrastructure configuration, you should use a command that relies on the local SMTP server instance, if available. Such a program is mailx. Solaris is bundled with a Sendmail instance and with the mailx program. When sending an email with mailx, it will internally invoke the local sendmail which must be properly configured in order to relay the message to the message destination.

Setting up SSH client access with TortoiseSVN

If you read so far, you're probably running your own Subversion repositories on your Solaris box. Fine! Now, let's face the next problem: some of your users needs access from a Windows client. They installed their favorite Subversion client, TortoiseSVN, and tried to checkout your repositories. But no, it does not work.

Setting up users

The first thing to do is setting up properly a bunch of user accounts for your client. If you haven't done yet, it's time to do it, now. Read here.

Preparing some keys for your users

As I told you in my previous post, the best option you have is setting up some public key for your users: configuration on the server side will be easier and your users won't need to enter a password every time they connect to your repositories. If you don't know how to do it, read this other post.

Configure TortoiseSVN

Many people get stuck here. Windows lacks the basic set of commands you need to interact with your remote system over an encrypted connection using the SSH protocol. It may sound strange to you faithful UNIX user but unfortunately that's the truth. Programs such as TortoiseSVN bring their own implementation of the SSH client, although specifically, TortoiseSVN lets you choose an alternate external client. The problem because of which people get stuck is that TortoiseSVN configuration GUI does no mention what-so-ever of SSH authentication. Nothing. That's why, once more, you should rely to The Manual just to discover that TortoiseSVN brings with it a PuTTY client, Plink, which is the command line interface to PuTTY backends.

The problem now reduces to configuring PuTTY to use a public key to authenticate you, save the session configuration and... remember its name! As Plink will use the same configuration registry as your standalone PuTTY, you'll be done.

Checkout your repository

Now that PuTTY is configured, you can check out your first repository over an SSH connection. Just remember not to use the server's URL and use PuTTY session name instead.

Now you should be able to:

• interact with your Solaris-hosted repositories from your Windows clients...
• ... with TortoiseSVN...
• ... and without typing any password.

Moreover, if you set up your Solaris user accounts as I explained you in another post, the key you distributed to your user won't let them even login into your system. You and you're sysadmin will be happy!

Configuring SSH key authentication with PuTTY

If you're a UNIX user, you're probably already using SSH public key authentication. Personally, I use it to avoid typing so many passwords every time I connect to a remote machine. If you're running a Windows client you installed an SSH client to connect to your remote machines. I usually use Cygwin, which gives me an environment very simiilar to what I'm used to. If you didn't feel like installing Cygwin just to establish a SSH connection, you probably chose PuTTY.

PuTTY is a bit different: it's got no .ssh directory to read from, it brings its SSH client implementation with it. If you want to configure PuTTY to use SSH key authentication, you can just follow these steps.

Setting up your keys for PuTTY

Both if you own or not your own keys, you need another program to produce a file for PuTTY to read: PuTTYgen. When you run it, puttygen will let you import your private key and save it in a PuTTY-friendly format or, if you haven't got one, to generate your brand new key. If you prefer not being asked a password by TortoiseSVN again and again, you can just avoid protecting the key with a passphrase and store the key in a safe place. I'll repeat it: store the key in a safe place.

Once you've done with the process, you will have a ppk file you should better store in a safe place! 

Configuring PuTTY

To tell PuTTY to use your key, just open it, go to Connection/SSH/Auth and browse for your key file in the Private key file for authentication... field. Now you can save your session and you're done.

Have fun!

Setting up SSH access to Subversion repositories on Solaris 10 (with zones)

If you followed my previous installments, you're probably running some Subversion repositories in your Solaris box. Chances are you're running the Subversion daemon, svnserve, and using the svn protocol to access the repositories: in a previous post I explained how you can set it up as an inetd service and in another post I gave you a pointer to an SMF manifests' repository where you can find a manifest to configure your Subversion daemon with the Solaris SMF framework.

There's another interesting way to access your Subversion repositories: tunneling the communication with svnserve over an encrypted SSH connection.

Why SSH?

There are many reasons that may lead you to such a choice and the most important might be:

• You're accessing your repositories from outside your network and want to use an encrypted connection.
• You don't want to maintain (yet another) user registry into the repository configuration files and you'd rather leverage your existing authentication strategy.

With SSH you can have existing users authenticate into your Solaris instances without additional effort. If you're already using some directory service, such as Sun Java System Directory Server, you already know the benefits of centralizing your user registry. If you're not, you should consider using a directory service before starting to duplicate sensible information such as user accounts, groups and privileges. If you're planning to give access to your repositories to users outside your organization, you could think a directory is not a good choice. Well, in this post I'll show you a possible workaround.

Tasks

To configure such a solution, you have to take into account the characteristics and the consequences of using SSH to authenticate your users into the Solaris Operating System. Whether you're using a directory service or local files (passwd, group, etc.), users who are going to connect must be managed at operating system level. You'll be able to give users access to your repositories without actually allowing them to perform any other operation on your system. Paranoid administrators shouldn't worry about users logging in into their machines, if they don't want to.

Subversion client configuration also allows you to fine tune the tunnel settings: you can change the binding port or the entire command itself, if you wish to.

You will also pay attention to the repositories' permissions: connecting via SSH, under some aspects, is just like using the local file protocol. Users connecting to your repositories, then, should have appropriate permissions on the repositories' directories.

You will also be able to leverage the Solaris Zones technology to isolate your Subversion repositories and users into a non global zone.

Configuring Subversion

This is pretty easy: if you're not fine tuning the tunnel definition, there's really nothing to configure. Just invoke the Subversion client using the svn+ssh schema and the job is done.

If you wished to fine tune your tunnel settings, you can edit the Subversion client config file. This file contains a section named [tunnels]. The config file is located in the .subversion subdirectory into your $HOME. If you want to change the default behavior associated to the svn+ssh schema, just edit (or create if it's missing) a line such as:

ssh = command

To change the default port, you could use:

ssh = ssh -p portnumber

If you wanted, you could also define your own schemas:

yourschema = yourcommand

would be used when accessing the repository with the svn+yourschema schema.

Another nice feauture of the configuration file is the possibility to override the tunnel definition with an environment variable. Defining a schema with the following syntax

yourschema = $YOURVAR yourcommand

has the following effect:

• If the variable $YOURVAR exists, it's used as the tunnel definition.
• If the variable doesn't exist, the tunnel definition provided in the configuration file is used instead.

The default value for the SSH tunnel definition is indeed the following:

ssh = $SVN_SSH ssh

When overriding the SSH tunnel definition you may choose to setup the $SVN_SSH variable for your users instead of modyfing the Subversion configuration file. We'll use this technique later.

Setting up the repository

One thing to take into account when using the SSH tunnel is that the svnserve command will be run with your user identity. This means that the user you're logging with must have proper permissions to access the repository files. The easiest way to go is probably creating a group for your users, let's say svn-group, and gave them write access to the repository directory, repository-dir:

$ chgrp -R svn-group repository-dir

$ chmod -R g+w repository-dir

If you're setting up multiple repositories you can create a group for each one of them. Please take into account that the Solaris operating systems allows an user to belong to a maximum of NGROUPS_MAX groups. If you also need to change the current group membership of an user because the required group is a Solaris secondary group, you can wrap the svnserve command into a script which changes the current user's group with the newgrp command.

Another good practice is setting a sane umask before accessing the repository files. You could wrap the svnserve command, or even the svn command if you're using the file schema too, into a shell script which sets the umask for the user:

#!/bin/sh
umask 002
# your commands here

Setting up public keys to use with SSH (and restrict user to only use Subversion)

When opening a SSH session, you're usually asked a password to authenticate into the remote machine. As explained in an earlier post, you can generate key pairs and use them for authentication. Key pairs also have another advantage: you can provide some users a key pair to authenticate and configure the SSH daemon to restrict their abilities to interact with the system. Specifically, you can setup the remote machine to only allow some users to launch a specific command, svnserve in this case, when authenticating. This is especially useful when you share a repository with users outside your organization. You can create user accounts and key pairs for them: with a proper configuration such users, although listed in your user database (both local files or directories such as LDAP), they will only be able to login and launch the svnserve command in tunnel mode. This approach together with Solaris Zone technology will give you the possibility:

• To quickly set up zones on your system to host subversion repositories.
• To optionally centralize your user accounts in the directory of your choice.
• To limit some user account to only use the Subversion server, effectively prohibiting them to open an interactive login session into your system.
• You can centralize the setup of the users' home directory by using the Solaris automounter. Subversion-only users will have their homes automounted from an ad-hoc server.

To configure the machine, or the Solaris zone, which hosts the Subversion server, you only have to follow the instructions in this post to provide them a key to connect to the server. Once this is done, if you want to limit your user ability by specifying a command to execute at login, you just have to add this fragment before the public part of the key:

command="/opt/csw/bin/svnserve -t"

In this case I specified svnserve path as installed by the Blastwave's package: if your setup is different, just change the path. If you're using Solaris Express Community Edition or OpenSolaris, Subversion may be found in:

command="/usr/bin/svnserve -t"


Please be aware that this fragment must be inserted before the key fragment and in the same line.

If the number of the users is such that you don't want to manage this process manually, you can for example:

• Use a script to generate the keys and to concatenate the public part into the authorized_keys2 file.
• Manage a centralized authorized_keys2 file.
• Share the authorized_keys2 file amongst users' home directories: they won't be able to read that file.
• Optionally automount users' home directories to share this configuration in many systems or zones.

Configure your users' groups

As mentioned earlier, users should belong to a group with the necessary permissions (read and write) on the repository directory. If you manage your users with local files, just assign them the proper primary and secondary roles: if you need secondary roles for some users you can use the newgrp command in a wrapper shell script to have an user login into the desired group before invoking the Subversion commands.

If you use a directory service, configure the directory appropriately. If you're using the Sun Java System Directory Server and you're using the default LDAP schema, assigning groups to user is pretty easy:

• The primary group can be specified setting the gidnumber attribute of the LDAP user entry.
• The secondary groups can be specified by adding multiple memberuid attributes into the group entry.

To add the users joe and john to a group you just add:

memberuid: joe

memberuid: john

into the group definition.

An alternative configuration with just one subversion user

If you do not want to leverage your existing user repository or you don't even have one, don't worry. In that case, all you have to do is setup your subversion repositories as usual and then manually setup the user for the current tunnel with the --tunnel-user=username option in the authorized_keys files. That's it.

Setting up zones

Setting up sparse zones in Solaris 10 is really straightforward and Sun Solaris 10 official Zones documentation covers the topic with great detail.

If you need to configure the zone to use an LDAP, please refer to the Solaris 10 Naming and Directory Services administration guide.

If you want to follow my advice, you can setup a zone for installing Blastwave's software and share the installation between zone using a loopback mount.

Next steps

You can now:

• Read this post about how to set up Subversion as an inted service.
• Read this post about how you can set up Solaris projects or zones to control resource usage.
• Read this post about how to set up Subversion post-commit hooks.
• Read this post about filtering Subversion commit using a post-commit hook.

If some of your users run Windows, you can read the following post to learn how to configure TortoiseSVN to use your public key to connect to Subversion using an SSH-tunneled connection.

Setting up Subversion post-commit hooks to send emails on Solaris 10

In previous post I explained how I set up some Subversions server on Solaris 10:

• Installing Subversion and setting it up as a Solaris 10 service
• Setting up Subversion as a Solaris 10 inetd service
  
• Setting up projects to control resource usage

Now I want to tell you how I set up Subversion post-commit hooks to automatically send emails after commit events.

Intercepting Subversion events

Subversion lets you hook programs to be executed when certain events are fired. The current version of Subversion lets you hook into:

• pre/post commit
• pre/post lock
• pre/post revprop-change
• pre/post unlock

If you have a look into your repository base directory, you should see a hooks directory probably filled up with templates. Depending on the platform the executable scripts naming may vary. In the case of Solaris, and an UNIX platform in general, you can drop in an executable or a shell script named post-commit and Subversion will execute it after a commit operation.

Post-commit hook parameters

Subversion passes the following positional parameters to the hook scripts:

• repository path
• revision number (after the commit)

If you want, you can use these parameters to customize the behavior of your hook script. I personally use them to customize the subject field and the contents of the email I send.

An example script

I always use a layer of indirection if it makes sense to me. In this case I chose to do the same and my example script just invokes another one, passing by the parameters received by Subversion. No secrets, here, just a one liner.

Sending emails

The problem now reduces to sending email from your host. This may depend on your system configuration and on your tastes. You could use mailx, mutt or even sendmail directly. The Subversion developers suggest using their mailer.py to send emails. Setting it up is pretty easy and you'll also find a configuration file and some tests on Subversion website.

Next steps

You can now:

• Read this post about how to set up Subversion as an inted service.
• Read this post about how you can set up Solaris projects or zones to control resource usage.
• Read this post about setting up SSH access to Subversion repositories.
• Read this post about filtering Subversion commit using a post-commit hook.

Setting up Subversion on Solaris 10 as an inetd service

Some posts ago I described how I set up Subversion on Solaris 10 in daemon mode and managed via SMF. It exists another way to invoke svnserve: it can be run via inetd. svnserve, when invoked with the -i|--inetd option will use stdin and stdout to communicate to a Subversion client: this is the expected behavior of an inetd-based service.

These instructions are valid for an UNIX system with an inetd implementation. Registration of this configuration as an SMF service is Solaris 10 specific.

Configuring /etc/services

The first thing to do is checking if /etc/services defines the Subversion service. IANA has reserved port 3690 for the Subversion service, so that /etc/services should contain:

svn 3690/tcp
svn 3690/udp

Please note that svn is just a service identifier which will be used during inetd configuration.

Setting up the inetd daemon

The standard UNIX inetd daemon accepts configuration on a file usually called /etc/inet/inetd.conf or /etc/inetd.conf for compatibility with BSD. In the case of Solaris 10, that file is not used anymore and inetd-based services are managed by SMF, too. There indeed is a /etc/inetd.conf file if you check your file system: it's used as a configuration file to be fed as input of the inetconv utility during the configuration of a service. inetconv can be invoked with the -i option to use an alternate file, which is what I usually do. Then, touch a new empty file or make a copy of /etc/inetd.conf which will be used later to configure the SMF service.

inetd.conf syntax

The expected syntax for /etc/inetd.conf file *or the file you'll feed inetconv with) is the following (see the man page for more details):

service-name endpoint-type protocol wait-status uid server-program server-arguments

• service-name: the name specified in the /etc/services file.
• endpoint-type: the type of the endpoint, which can be stream, dgram, raw, seqpacket, tli.
  
• protocol: a valid protocol entry in the /etc/inet/protocols file. Additionally, tcp6 and udp6 may also be used. In the case of RPC services, it's a string of the type rpc/* or rpc/..., where ... is a combination of nettypes and/or netids.
  
• wait-status: this field may assume the values wait or nowait. Basically, it instructs inetd whether to pass over the listening socket to the service process and waiting for it to finish before resuming listening.
  
• uid: the user id under which the service will run
  
• server-program: the pathname of the server process executable or an internal name if inetd itself will provide this service.
  
• server-arguments: the entire command line arguments for the service process, including the service process name itself (argument 0)
  

Subversion entry

The syntax to use for svnserve is the following:

svn stream tcp nowait svnuser /opt/csw/bin/svnserve svnserve -i -r /home/svnuser/svnrepos

• I previously set up a standard user, svnuser, under which the Subversion service will run.
  
• In this case I'm using the Subversion package installed from Blastwave which installs into /opt/csw/bin.
• The repositories served by this instance are located into directory /home/svnuser/svnrepos.

Setting up the Subversion user

Creating the user

Subversion server process will run as a regular user called svnuser. This user is a basic user created with:

# useradd -d /export/home/svnuser -m -k /etc/skel -s /bin/bash svnserve
# passwd svnuser
[...]

Assuming that home automounting under /home is configured, I then modify the user with:

# usermod -d /home/svnuser svnuser

In my case, this user was created in the LDAP directory which I manage with Sun Java System Directory Server Enterprise Edition but basically the result is equivalent to running the above mentioned commands.

Creating a default user project (Solaris 10 specific)

I also created an user project for this user, called user.svnuser, so that processes launched after svnuser login will be associated with its project. The project I created just caps the CPU associated with its processes:

# projadd -K "project.cpu-cap=(privileged,50,deny)" user.svnserve

Setting up the SMF service (Solaris 10 specific)

After setting up inetd, under Solaris 10 we have to configure the described service with inetconv. inetconv it's an utility which reads files containing inetd.conf-like records and maps them to auto-generated SMF services. If the file we've generated it's called svn.conf, we have to run the following command:

# inetconv -i svn.conf

and a new SMF service description will be generated, imported and enabled.

Conclusion

Setting up inetd based services it's even easier with Solaris 10. Moreover, the powerful SMF framework will allow the sysadmin to manage and observe every service and enjoy the advantages of SMF. The only problem not (yet) addressed by this blog post is that a nowait inetd service is fork()-ed and exec()-ed by inetd and as Solaris 10 documentation states, during the fork() system call the process and task controls are inherited. This has the effect that even if svnserve is fork()-ed and then run under the svnuser account, the project.cpu-cap entry is having no effect. To discover how to configure an execution method of an SMF-managed service to run into a Solaris 10 project, read this post.

Next steps

You can now:

• Read this post about how you can set up Solaris projects or zones to control resource usage.
• Read this post about setting up SSH access to Subversion repositories.
• Read this post about how to set up Subversion post-commit hooks.
• Read this post about filtering Subversion commit using a post-commit hook.

Setting up Subversion on Solaris 10

In this series of posts I'm going to describe how I set up Subversion on a Solaris 10 machine. The instructions here are valid for subsequent releases such as Solaris Express Community Edition (Nevada) and OpenSolaris (Indiana), the only difference being the fact that Subversion is bundled with the Operating System.

I decided to host some Subversion (1.4.5) repositories on a Solaris 10 (update 5) server so I downloaded Subversion packages from Blastwave and installation was done in less than 5 minutes with a simple:

# pkg-get -i subversion

Once Blastwave's installer was done, the first thing I wanted to do was registering Subversion's daemon (svnserve) in the Service Management Facility. Writing an SMF manifest is not so hard but fortunately I found a manifest and a management script in OpenSolaris website and the job was done! Great work, guys.

Next steps

You can now:

• Read this post about how to set up Subversion as an inted service.
• Read this post about how you can set up Solaris projects or zones to control resource usage.
• Read this post about setting up SSH access to Subversion repositories.
• Read this post about how to set up Subversion post-commit hooks.
• Read this post about filtering Subversion commit using a post-commit hook.