In this article, I'll show an example of how to implement a Subversion pre-commit hook to integrate with Drupal. Pre-commit hooks can be used to execute any arbitrary code, such as deployment procedures, archiving databases, etc. For this example, I will show how to check for the creation of a subversion tag and archive the database.

To get started, I created a local subversion repository.

# create folder for subversion repositories
$ mkdir /var/subversion

# create the subversion repository
$ svnadmin create /var/subversion/project

Upon creating a new local svn repository, a hooks directory will be created. Example: /var/subversion/project/hooks

Inside this directory will be a bunch of sample scripts ending in ".tmpl" which contain example hook scripts. Here are the contents of the example pre-commit hook without comments:

$ cat pre-commit.tmpl | egrep -iv "(^#|^$)"
REPOS="$1"
TXN="$2"
SVNLOOK=/usr/bin/svnlook
$SVNLOOK log -t "$TXN" "$REPOS" | \
   grep "[a-zA-Z0-9]" > /dev/null || exit 1
commit-access-control.pl "$REPOS" "$TXN" commit-access-control.cfg || exit 1
exit 0

As noted in the pre-commit.tmpl file, there are 2 arguments being passed to the pre-commit script:

[1] REPOS-PATH   (the path to this repository)
[2] TXN-NAME     (the name of the txn about to be committed)

I created a new file called "pre-commit" and added the following contents:

#!/bin/bash

/var/www/vhosts/project.vm/scripts/svn-pre-commit.php "$1" "$2"

I then made the file executable.

$ chmod ug+w pre-commit

The above script simply passes the arguments to a PHP script contained with the Drupal project.

In my scripts folder (/var/www/vhosts/project.vm/scripts), I created the PHP script "svn-pre-commit.php" with the following contents:

#!/usr/bin/php
<?php

// get args
$repo = $argv[1];

// define path to mysql backups
$mysql_backups_path = '/var/www/vhosts/project.vm/database';

// define path to drupal docroot
$drupal_docroot_path = '/var/www/vhosts/project.vm/htdocs';

// get changed path
// example output:
// A   tags/20110503/
$svn_look = `svnlook changed $repo`;

// define pattern to break apart svnlook changed
$pattern = '/^\s*([A-Za-z])\s*(.*)$/';

// execute preg match
preg_match($pattern, $svn_look, $matches);
$svn_action = $matches[1];
$svn_changed_path = $matches[2];

// check if a tag is being created
if ($svn_action == 'A' && substr($svn_changed_path, 0, 5)=='tags/') {

  // get tag name
  $exploded = explode('/', $svn_changed_path);
  $tag_name = $exploded[1];
  
  // change dir to drupal docroot
  chdir($drupal_docroot_path);

  // backup mysql database using drush
  `/var/www/drush/drush sql-dump > {$mysql_backups_path}/tag_{$tag_name}.sql`;

}

I also made this script executable:

$ chmod ug+w svn-pre-commit.php

Now, assuming that my Drupal site is integrated with the subversion repository, and development is at a point to deploy/create a new tag, I executed the following command to create the subversion tag:

$ svn cp file:///var/subversion/project/trunk file:///var/subversion/project/tags/beta-0.1 -m "creating beta 0.1 tag"

To verify, I entered the directory containing my database dumps to checkout the result:

$ cd /var/www/vhosts/project.vm/database

$ ls -1
tag_beta-0.1.sql

In certain situations you need more control of your cron tasks, execution times, and methods. Check out the SuperCron and Elysia Cron modules as alternative solutions. Both give you the ability to order, manage, disable, and execute your cron tasks, among a slew of other functionality. If you are familiar with Linux crontabs and need that level of scheduling and control, see Elysia Cron.

On a recent project, I struggled to get a massive cron hook to complete, and at one point, some of the tasks required a Batch API implementation to manage system resources. I eventually decided to pull the cron task out of the Drupal/Apache/web environment to execute it on the shell in a separate crontab. Here's the gist of the PHP script I put in my Drupal docroot.

<?php

// check if this is web traffic, versus CLI
if (isset($_SERVER['HTTP_USER_AGENT'])) {
 header('HTTP/1.1 403 Forbidden');
 die('You are not authorized to access this page.');
}

// set environment variables:

// remove time limit of script
set_time_limit(0);

// increase memory usage
ini_set('memory_limit', '256M');

// ensure script is being executed from Drupal docroot
$docroot_path = dirname($_SERVER['SCRIPT_NAME']);
if (getcwd() != $docroot_path) {
 chdir($docroot_path);
}

// set bootstrap variables:
// these lines ensure multi-site environments will work with a bootstrap
$_SERVER['HTTP_HOST'] = 'www.myhostname.com';
$_SERVER['SCRIPT_NAME'] = '/' . basename(__file__);

// include Drupal bootstrap
require_once './includes/bootstrap.inc';
drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);

// check for custom module and cron hook
if (!module_exists(array('MYMODULE')) || !function_exists('MYMODULE_cron')) {
 die('Module is not enabled.');
}

// execute custom cron hook
MYMODULE_cron();

?>

I then added a line in my crontab on the shell

$ crontab -e

# file contents:

#min hour dayMonth month dayWeek command

0    0    *        *     *       /usr/bin/php /path/to/drupal/docroot/cron.custom.php

The above crontab executes my custom cron hook once a day.

My Drupal photo gallery implements a total of 9 imagecache presets, but due to a PHP memory_limit cap of 90MB in my previous hosting contract, I frequently encountered issues generating images.

Since I was on a shared hosting plan, PHP was executed as CGI and a service ran on the server to kill PHP scripts that reached the PHP memory_limit. This resulted in the following error, which prevented imagecache from creating missing images and redirecting the user properly.

Premature end of script headers: index.php

After changing my hosting plan and increasing my server resources, I decided to write a PHP script to programmatically generate all the missing images. This script when executed on the shell using Drush, will get a list of every image in the files directory, and make an HTTP request for each imagecache preset URL.

<?php

// store original working directory
define('ORIGINAL_DIRECTORY', getcwd());

// define site http host
define('HTTP_HOST','pics.ericlondon.com');

// get imagecache presets data
$presets_data = imagecache_presets();

// loop through presets data and collect preset names
$presets = array();
foreach ($presets_data as $key => $value) {
  $presets[] = $value['presetname'];
}

// get a list of pictures
chdir(file_directory_path());
// NOTE: the following line uses find, grep, and sed to generate a list of image files.
// It will vary depending on which file extensions to include and which directories to ignore
$command = 'find . -type f | egrep -i "\.(gif|jpeg|jpg|png)$" | sed "s/^\.\///" | egrep -iv "^(imagecache|imagefield_thumbs)\/"';
$output = `$command`;
$files = explode("\n", trim($output));
chdir(ORIGINAL_DIRECTORY);

// loop through file list
foreach ($files as $file) {

  // loop through each preset
  foreach ($presets as $preset) {
  
    // define url
    $url = 'http://' . HTTP_HOST . '/' . file_directory_path() . '/imagecache/' . $preset . '/' . $file;
    
    // request url
    $http_request_data = drupal_http_request($url);
    
    // log entry
    $log_entry = $http_request_data->code . " " . $http_request_data->status_message . " " . $url;
    file_put_contents('ic_preset_log.txt', $log_entry . "\n", FILE_APPEND);
  
  }

}
?>

I executed this script on the shell using the following drush command:

drush scr generate_presets.php

The script logs every request and can be reviewed afterward..

$ tail ic_preset_log.txt 
200 OK http://pics.ericlondon.com/sites/ericlondon.com/files/imagecache/large_landscape/userpics/10020/118_1881.JPG
200 OK http://pics.ericlondon.com/sites/ericlondon.com/files/imagecache/large_portrait/userpics/10020/118_1881.JPG
200 OK http://pics.ericlondon.com/sites/ericlondon.com/files/imagecache/medium/userpics/10020/118_1881.JPG
200 OK http://pics.ericlondon.com/sites/ericlondon.com/files/imagecache/medium_landscape/userpics/10020/118_1881.JPG
200 OK http://pics.ericlondon.com/sites/ericlondon.com/files/imagecache/medium_portrait/userpics/10020/118_1881.JPG
200 OK http://pics.ericlondon.com/sites/ericlondon.com/files/imagecache/thumbnail/userpics/10020/118_1881.JPG
200 OK http://pics.ericlondon.com/sites/ericlondon.com/files/imagecache/thumbnail_landscape/userpics/10020/118_1881.JPG
200 OK http://pics.ericlondon.com/sites/ericlondon.com/files/imagecache/thumbnail_portrait/userpics/10020/118_1881.JPG
200 OK http://pics.ericlondon.com/sites/ericlondon.com/files/imagecache/large/userpics/10020/2007_07_21_003.JPG
200 OK http://pics.ericlondon.com/sites/ericlondon.com/files/imagecache/large_landscape/userpics/10020/2007_07_21_003.JPG

Sometimes there's no escaping hard coded hostnames being stored in your Drupal database. If you have ever setup language translations (i18n) in production, and then deployed the site to a development environment (with a different hostname), you'll know exactly what I am talking about. The first thing you'll notice in your dev site is that all your urls are linking back to the production site, and you may even have a hard time logging in.

If you query the languages table, you'll see the following:

select language, domain
from {languages}

Results:

language	domain
de		http://de.ericlondon.com
en		http://www.ericlondon.com
es		http://es.ericlondon.com
fr		http://fr.ericlondon.com

At first I thought it would make sense to write a module or a hook_update_n() function to resolve this problem when deploying to another environment, but this method seemed rather clunky, repeatedly running the same revision of a module's update script. In the end, I decided to simply write a PHP script to run via Drush. The following code fetches your default language and all records in the language table, loops through them, and updates the hostname records to match your new hostname...

<?php

// ensure this script is being called from Drush ONLY
// ensure HTTP_HOST is being used
if (
  !is_array($_SERVER['argv'])
  || substr($_SERVER['argv'][0], -9) != 'drush.php'
  || substr($_SERVER['SCRIPT_FILENAME'], -9) != 'drush.php'
  || !strlen($_SERVER['HTTP_HOST'])
) {
  header('HTTP/1.1 403 Forbidden');
  die('Access Denied.');
}

$output = "";

// fetch hostname from globals
$http_host = $_SERVER['HTTP_HOST'];

// ensure http host exists
if (!$http_host || !is_string($http_host)) {
  die('Error occurred fetching http host.');
}

// set default http host
$default_http_host = 'www.' . $http_host;

// define default language
$default_language = 'en';

// define SQL to fetch default language
$sql = "
  select *
  from {languages}
  where language = '%s'
";

// execute SQL
$default_language_record = db_fetch_object(db_query($sql, $default_language));

// ensure record exists
if (!is_object($default_language_record)) {
  die('Error occurred fetching default language.');
}

// update default language record
$default_language_record->domain = 'http://' . $default_http_host;

// set new variable for "language_default"
variable_set('language_default', $default_language_record);
$output .= "Setting language_default variable to: " . $default_language_record->domain . "\n";

// fetch language records
$sql = "
  select *
  from {languages}
";
$resource = db_query($sql);
$language_records = array();
while ($row = db_fetch_object($resource)) {
  $language_records[] = $row;
}

// correct language records
foreach ($language_records as $key => $value) {

  // determine language prefix  
  switch ($value->language) {
    case $default_language:
      $prefix = 'www.';
      break;
    default:
      $prefix = $value->language . '.';
      break;
  }

  $new_domain = 'http://' . $prefix . $http_host;

  $sql = "
    update {languages}
    set domain = '%s'
    where language = '%s'
  ";
  db_query($sql, $new_domain, $value->language);

  $output .= "Setting language record for language: \"" . $value->language . "\" to: " . $new_domain . "\n";

}

drupal_flush_all_caches();

$output .= "Executed: drupal_flush_all_caches()\n";

$output .= "DONE.\n";

echo $output;
?>

I dropped this script (named: fix_env.php) in the root of my Drupal site, and then executed it via Drush using the following shell command:

drush --uri=http://tdb.erl.dev scr fix_env.php

The "--uri" flag allows you to specific a hostname for multi-site environments, and the "scr" flag allows you to specify a php script to execute.

NOTE: I'm in the habit of never using sites/ericlondon.com, because it's a real PITA if you ever try to move your site into a multi-site configuration ^_^

After running the script via Drush, my hostname database records have been corrected to match my development environment...

select language, domain
from {languages}

Results:

language	domain
de		http://de.tdb.erl.dev
en		http://tdb.erl.dev
es		http://es.tdb.erl.dev
fr		http://fr.tdb.erl.dev

In this tutorial, I'll show how you can use awk, grep, and sed (my favorite command line tools) to backup and archive your MySQL databases. This can be useful to schedule a cron job, transfer your databases to another server, or any other type of scripting.

First, you'll have to get acquainted with connecting to and dumping your database on the command line. Depending on your user, credentials, and where the databases are located, your command might look something like this. Please note, there is no space between the password and the "-p" flag.

$ mysqldump -u user -pPASSWORD -h hostname database > database.sql

To simplify my example, I'm going to shorten the mysqldump command to the follow.

$ mysqldump database > database.sql

Now that we're MySQL command line pros, I'll break down each command. I'll start by showing all the databases.

Eric-Londons-MacBook-Pro:backup Eric$ mysql --execute="show databases"
+---------------------+
| Database            |
+---------------------+
| customers           |
| db_pics_ericlondon  |
| db_thedrupalblog_d6 |
| drupal              |
| drupal-pics         |
| drupalmusicproject  |
| itunes              |
+---------------------+

Now, I'll "pipe" the output from the previous command into awk to show the first column data.

Eric-Londons-MacBook-Pro:backup Eric$ mysql --execute="show databases" | awk '{print $1}'
Database
customers
db_pics_ericlondon
db_thedrupalblog_d6
drupal
drupal-pics
drupalmusicproject
itunes

And use grep to remove the first line that says "Database".

Eric-Londons-MacBook-Pro:backup Eric$ mysql --execute="show databases" | awk '{print $1}' | grep -iv ^Database$
customers
db_pics_ericlondon
db_thedrupalblog_d6
drupal
drupal-pics
drupalmusicproject
itunes

And use sed to build the mysqldump command. This one is kinda tricky, sorry. As you can see, I also embedded the date command in there to generate today's date in the format: YYYYMMDD.

Eric-Londons-MacBook-Pro:backup Eric$ mysql --execute="show databases" | awk '{print $1}' | grep -iv ^Database$ | sed 's/\(.*\)/mysqldump \1 > \1.'$(date +"%Y%m%d")'.sql/'
mysqldump customers > customers.20100825.sql
mysqldump db_pics_ericlondon > db_pics_ericlondon.20100825.sql
mysqldump db_thedrupalblog_d6 > db_thedrupalblog_d6.20100825.sql
mysqldump drupal > drupal.20100825.sql
mysqldump drupal-pics > drupal-pics.20100825.sql
mysqldump drupalmusicproject > drupalmusicproject.20100825.sql
mysqldump itunes > itunes.20100825.sql

Lastly, if everything looks good, you can pipe the output back to the command line.

Eric-Londons-MacBook-Pro:backup Eric$ mysql --execute="show databases" | awk '{print $1}' | grep -iv ^Database$ | sed 's/\(.*\)/mysqldump \1 > \1.'$(date +"%Y%m%d")'.sql/' | sh

Eric-Londons-MacBook-Pro:backup Eric$ ls -1
customers.20100825.sql
db_pics_ericlondon.20100825.sql
db_thedrupalblog_d6.20100825.sql
drupal-pics.20100825.sql
drupal.20100825.sql
drupalmusicproject.20100825.sql
itunes.20100825.sql

You could even take this one step further and pipe the output through gzip to compress the dumps :)