svn | Eric London's Blog

Scanning a file system path for all version control remote repository urls

Submitted by Eric.London on 12/20/2011

Tags:

Add new comment

Just thought I'd share a PHP shell script to scan a file system path, search for all .git/.svn directories recursively, and collect a unique list of all remote repository URLs. It uses "svn info" or "git remote" to get the repository URL path.

<?php
// define a path to scan
$scan_path = '/var/www/vhosts';

// keep track of current working dir
$original_cwd = $_SERVER['PWD'];

// get a list of all .git and .svn directories
$files = trim(`find "$scan_path" -type d | egrep -ir '\.(git|svn)$'`);

// explode on "\n"
$files = explode("\n", $files);

// loop through files
$repo_list = array();
foreach($files as $key => $file) {

  // .git or .svn ?
  $repo_type = substr($file, -4);
  $repo_path = substr($file, 0, -4);

  switch ($repo_type) {
    case '.svn':

      // get svn repo root
      $repo_root = trim(`svn info "$repo_path" | grep ^Repository\ Root | sed 's/Repository Root: //'`);
      if (!in_array($repo_root, $repo_list)) {
        $repo_list[] = $repo_root;
      }
      break;

    case '.git':

      // change dir
      chdir($repo_path);
      
      // get git remote path
      $repo_root = trim(`git remote -v | grep -ir fetch | awk '{print \$2}' | head -1`);
      if (!in_array($repo_root, $repo_list)) {
        $repo_list[] = $repo_root;
      }
      break;
  }

}

// sort
sort($repo_list);

// go back to cwd
chdir($original_cwd);

// output repo list into file
file_put_contents('repo_list.txt', implode("\n", $repo_list) . "\n");
?>

I put this PHP in a file called "scan-for-version-control.php", and run it by typing:

$ php scan-for-version-control.php

It created a file in the current working directory: repo_list.txt containing stuff like..

git@myuser.someversioncontrolhost.com:myuser/somerepo1.git
git@myuser.someversioncontrolhost.com:myuser/somerepo2.git
https://myuser.svn.someversioncontrolhost.com/somerepo1
https://myuser.svn.someversioncontrolhost.com/somerepo2

Implementing a Subversion pre-commit hook to integrate with Drupal

Submitted by Eric.London on 5/8/2011

Tags:

Add new comment

In this article, I'll show an example of how to implement a Subversion pre-commit hook to integrate with Drupal. Pre-commit hooks can be used to execute any arbitrary code, such as deployment procedures, archiving databases, etc. For this example, I will show how to check for the creation of a subversion tag and archive the database.

To get started, I created a local subversion repository.

# create folder for subversion repositories
$ mkdir /var/subversion

# create the subversion repository
$ svnadmin create /var/subversion/project

Upon creating a new local svn repository, a hooks directory will be created. Example: /var/subversion/project/hooks

Inside this directory will be a bunch of sample scripts ending in ".tmpl" which contain example hook scripts. Here are the contents of the example pre-commit hook without comments:

$ cat pre-commit.tmpl | egrep -iv "(^#|^$)"
REPOS="$1"
TXN="$2"
SVNLOOK=/usr/bin/svnlook
$SVNLOOK log -t "$TXN" "$REPOS" | \
   grep "[a-zA-Z0-9]" > /dev/null || exit 1
commit-access-control.pl "$REPOS" "$TXN" commit-access-control.cfg || exit 1
exit 0

As noted in the pre-commit.tmpl file, there are 2 arguments being passed to the pre-commit script:

[1] REPOS-PATH   (the path to this repository)
[2] TXN-NAME     (the name of the txn about to be committed)

I created a new file called "pre-commit" and added the following contents:

#!/bin/bash

/var/www/vhosts/project.vm/scripts/svn-pre-commit.php "$1" "$2"

I then made the file executable.

$ chmod ug+w pre-commit

The above script simply passes the arguments to a PHP script contained with the Drupal project.

In my scripts folder (/var/www/vhosts/project.vm/scripts), I created the PHP script "svn-pre-commit.php" with the following contents:

#!/usr/bin/php
<?php

// get args
$repo = $argv[1];

// define path to mysql backups
$mysql_backups_path = '/var/www/vhosts/project.vm/database';

// define path to drupal docroot
$drupal_docroot_path = '/var/www/vhosts/project.vm/htdocs';

// get changed path
// example output:
// A   tags/20110503/
$svn_look = `svnlook changed $repo`;

// define pattern to break apart svnlook changed
$pattern = '/^\s*([A-Za-z])\s*(.*)$/';

// execute preg match
preg_match($pattern, $svn_look, $matches);
$svn_action = $matches[1];
$svn_changed_path = $matches[2];

// check if a tag is being created
if ($svn_action == 'A' && substr($svn_changed_path, 0, 5)=='tags/') {

  // get tag name
  $exploded = explode('/', $svn_changed_path);
  $tag_name = $exploded[1];
  
  // change dir to drupal docroot
  chdir($drupal_docroot_path);

  // backup mysql database using drush
  `/var/www/drush/drush sql-dump > {$mysql_backups_path}/tag_{$tag_name}.sql`;

}

I also made this script executable:

$ chmod ug+w svn-pre-commit.php

Now, assuming that my Drupal site is integrated with the subversion repository, and development is at a point to deploy/create a new tag, I executed the following command to create the subversion tag:

$ svn cp file:///var/subversion/project/trunk file:///var/subversion/project/tags/beta-0.1 -m "creating beta 0.1 tag"

To verify, I entered the directory containing my database dumps to checkout the result:

$ cd /var/www/vhosts/project.vm/database

$ ls -1
tag_beta-0.1.sql

Merging a subversion branch back into trunk

Submitted by Eric.London on 1/30/2010

Tags:

3 comments

Here's an tutorial explaining how to merge a subversion branch back into trunk:

# First you'll need a local working copy of your trunk
$ cd /path/to/checkout/trunk
$ svn co https://HOSTNAME/repo/trunk .

# The above command will return the latest revision of the trunk
# which you'll need to remember for a later command. Example:
Checked out revision 76.

# If you already have a working copy of your trunk,
# running an "svn update" will set your repo to the latest revision
# and return the latest revision:
$ cd /path/to/checkout/trunk
$ svn update
At revision 76.

# Now, you'll need to find the revision when the branch was created:
$ svn log --stop-on-copy https://HOSTNAME/repo/branches/20091205 | tail -5
------------------------------------------------------------------------
r63 | eric | 2009-10-07 15:10:28 -0400 (Wed, 07 Oct 2009) | 1 line

Created branch.
------------------------------------------------------------------------

# Now that you know the latest trunk revision and the branch created revision,
# you can execute the merge on your local working copy of trunk
# NOTE: it would be a good idea to ensure you are not executing the merge
# in an active environment!
$ cd /path/to/checkout/trunk
$ svn merge -r 63:76 https://HOSTNAME/repo/branches/20091205 .

# NOTE: the revision range syntax is: "-r BRANCH:TRUNK"

# Now if you execute an "svn stat" command you'll see that all the changes from your branch have been applied to your working copy of trunk. Hopefully, there are no conflicts! Now you can review the changes, test your code, and commit.

If you need to merge some changes made to trunk into your branch, you could do the following:

# find the branch created revision:
$ svn log --stop-on-copy https://HOSTNAME/repo/branches/20091205 | tail -5
------------------------------------------------------------------------
r63 | eric | 2009-10-07 15:10:28 -0400 (Wed, 07 Oct 2009) | 1 line

Created branch.
------------------------------------------------------------------------

# find the latest revision of trunk:
$ svn info https://HOSTNAME/repo/trunk | grep -i ^Last\ Changed\ Rev
Last Changed Rev: 76

# from checked out local copy of branch:
$ svn merge -r 63:76 https://HOSTNAME/repo/trunk .

Installing subversion on a Mac and handling differences in Subversion repository versions

Submitted by Eric.London on 9/15/2009

Tags:

Add new comment

When I got my new MacBook Pro, I installed Xcode which comes with subversion (version 1.6.x):

$ which svn
/usr/bin/svn

$ /usr/bin/svn --version | head -1
svn, version 1.6.2 (r37639)

After installing Xcode I checked out some repositories to my local filesystem. Soon afterward, I realized I needed to be running an older subversion client to stay compatible with some 1.5.x repositories, so I decided to install CollabNet's OSX subversion binary (registration is required for older releases).

After downloading and installing the package (which defaults to /opt/subversion/bin/svn), I edited the /etc/profile file to override priority of my $PATH variable (since I now had 2 versions of subversion installed):

# lines added to /etc/profile:
export PATH=/opt/subversion/bin:$PATH

Now, if I ran a "which" command for svn, the appropriate svn executable is returned:

$ which svn
/opt/subversion/bin/svn

Unfortunately, the subversion projects I checked out using Xcode's subversion were inaccessible due to differences in the .svn structure:

$ cd /path/to/my/1.6.x/repo

$ svn stat
svn: This client is too old to work with working copy '.'.  You need
to get a newer Subversion client, or to downgrade this working copy.
See http://subversion.tigris.org/faq.html#working-copy-format-change
for details.

Luckily, CollabNet has a downloadable python script which allows you to switch checked out repositories to different versions. I downloaded this file and copied it into /opt/subversion/bin.

I was now able to update/downgrade my repositories using this python script:

$ svn --version | head -1
svn, version 1.5.7 (r36142)

$ cd /path/to/my/1.6.x/repo

$ change-svn-wc-format.py . 1.5
Converted WC at '.' into format 9 for Subversion 1.5

Updating a Drupal module in a subversion integrated filesystem using rsync

Submitted by Eric.London on 8/30/2009

Tags:

7 comments

Version control is an essential tool when it comes to maintaining your code and properly tracking filesystem changes. In this quick tutorial, I'll show you how to update a Drupal module in a subversion integrated environment using rsync. Since simply copying the contents of a new module update on top of your current directory structure is a bad idea (since it will NOT account for file deletions), rsync is a great solution for syncing module update changes. NOTE: it is a bad idea to update a module in a production environment without proper testing; this code assumes you are working in a development environment.

The first step is to download the latest (and in most cases, stable) package for the module you'd like to update to a directory outside your drupal path. For my Drupal installations, I browse to the Available updates page (admin/reports/updates) and copy the URL from this page. Next I go to my shell, use wget to fetch the package to my home directory, and unpack the file:

$ cd ~
$ mkdir Downloads
$ cd Downloads
$ wget http://ftp.drupal.org/files/projects/xmlsitemap-6.x-2.x-dev.tar.gz
$ tar -xzf xmlsitemap-6.x-2.x-dev.tar.gz

Now you can run the rsync command to apply the filesystem changes to your Drupal module directory. You'll have to update the paths listed below to match your filesystem and Drupal installation.

$ rsync -avCz --delete ~/Downloads/xmlsitemap/ /var/www/vhosts/tdb.erl.dev/httpdocs/sites/all/modules/xmlsitemap/

If executed properly, you can change directory to the module you are trying to update and run an "svn stat" command to see what has been updated. NOTE: the below output is simulated to show common svn status changes.

$ cd /var/www/vhosts/tdb.erl.dev/httpdocs/sites/all/modules/xmlsitemap
$ svn stat
?      xmlsitemap.newfile.php
!      xmlsitemap_taxonomy
M      xmlsitemap.module

At this point, you should see a list of all the files that have changed with this module update. You can now run the update.php script in your development environment and verify the changes are working properly. If everything is working properly, you can commit the module update changes...

# remove files that have been deleted
$ svn stat | grep ^! | awk {'print $2'} | xargs -i svn rm '{}'

# add files that have been added
$ svn stat | grep ^? | awk {'print $2'} | xargs -i svn add '{}'

# commit
$ svn commit -m "upgraded xmlsitemap module to version 6.x-2.x-dev"

Now, you can safely deploy the module changes to your production environment (svn update), run the update.php script, and ensure everything is working properly.