In this article I'll explain how I recently setup a web server to host both 1. Ruby on Rails via Phusion Passenger (mod_rails), and 2. PHP via Apache (mod_php). Nginx will sit in front and proxy requests (by hostname) to Apache, or serve them directly via Phusion. Here's a rough diagram:



I started with a fresh (minimal) installation of Ubuntu 10.04 LTS. Here we go..

# update installed packages
$ sudo apt-get update
$ sudo apt-get upgrade

# install SSH server
$ sudo apt-get install openssh-server -y

Part 1, Apache/PHP

# install PHP & Apache
sudo apt-get install php5 php5-cli php5-common php5-curl php5-gd php5-mysql php-pear -y

Set Apache to listen on port 8000.
Note: nginx will listen on 80 and proxy requests to Apache.

# edit file: /etc/apache2/ports.conf

# replace:
NameVirtualHost *:80
Listen 80

# with:
NameVirtualHost *:8000
Listen 8000

For sake of this tutorial, I created a simple PHP script.

$ mkdir /var/www/php.eric.vm
$ echo '<?php echo "hello php world";' >> /var/www/php.eric.vm/index.php

And created an Apache vhost for the above script:

# created new/example file: /etc/apache2/sites-available/php.eric.vm

<VirtualHost *:8000>

  ServerName php.eric.vm
  ServerAdmin webmaster@localhost
  DocumentRoot /var/www/php.eric.vm

  <Directory /var/www/php.eric.vm >
    AllowOverride All
  </Directory>

  ErrorLog /var/log/apache2/php.eric.vm-error.log

  LogLevel warn

  CustomLog /var/log/apache2/php.eric.vm-access.log combined

</VirtualHost>

# Enabling the new conf file by adding a symlink:
$ cd /etc/apache2/sites-enabled
$ sudo ln -s ../sites-available/php.eric.vm

# removed the existing default vhost:
$ sudo rm 000-default

# restarted Apache
$ sudo service apache2 restart

At this point, I was able to reach my php script by browsing to http://php.eric.vm:8000

Part 2, RVM/Ruby/Passenger

# Install CURL, to start the RVM installation
sudo apt-get install curl -y

# Install RVM (multi-user installation)
$ sudo bash -s stable < <(curl -s https://raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer )

# add user to RVM group
$ sudo usermod -a -G rvm eric

# Install Ruby/RVM dependencies
# NOTE: you can run "rvm requirements" to get this list:
$ sudo apt-get install build-essential openssl libreadline6 libreadline6-dev curl git-core zlib1g zlib1g-dev libssl-dev libyaml-dev libsqlite3-0 libsqlite3-dev sqlite3 libxml2-dev libxslt-dev autoconf libc6-dev ncurses-dev automake libtool bison subversion -y

# install nodejs, per javascript runtime
$ sudo apt-get install python-software-properties
$ sudo add-apt-repository ppa:chris-lea/node.js
$ sudo apt-get update
$ sudo apt-get install nodejs

# install ruby 1.9.2
$ rvmsudo rvm install 1.9.2

# set default version of ruby
$ rvm use 1.9.2 --default

# install rails
$ rvmsudo gem install rails --version 3.2.1

# install mysql server
# note: rails defaults to sqlite3, choose whatever you want
$ sudo apt-get install mysql-server -y

# install passenger
$ rvmsudo gem install passenger

# install nginx/passenger requirements
$ sudo apt-get install libcurl4-openssl-dev -y

# install passenger nginx module
$ rvmsudo passenger-install-nginx-module
# Install options I chose:
# 1. Yes: download, compile and install Nginx for me. (recommended)
# Please specify a prefix directory [/opt/nginx]: 

Part 3, Test Rails App

For this tutorial, I created a (very) simple Rails app.

# create new rails app
$ cd /var/www
$ rails new railsdemo
$ cd railsdemo

# integrate with git version control
$ git init
$ git add .
$ git commit -am "initial rails repo"

# remove default placeholder index page
$ rm public/index.html

# create sample home controller
$ rails generate controller home index

# add route in file: config/routes.rb
root :to => "home#index"

# version control
$ git add .
$ git commit -am "added home controller and route"

(as usual) if I had made changes to my models, I would have run:

$ rake db:migrate

To test my rails development environment:

$ rails s

At this point, I was able to browse to my rails app at: http://rails.eric.vm:3000
The generic controller message was shown:
Home#index
Find me in app/views/home/index.html.erb

To run the rails app in production mode, I edited the file: config/environments/production.rb, and made this change:

config.assets.compile = true

And as necessary, migrate production database:

$ RAILS_ENV=production rake db:migrate

At this point, the rails app should be able to run in production mode using:

$ rails s -e production

(if not, check log/production.log for errors)

Part 4, Nginx

Although nginx is now installed, you'll need a init script. I simply copied the one listed here: http://techoctave.com/c7/posts/16-how-to-host-a-rails-app-with-phusion-passenger-for-nginx, and pasted it here: /etc/init.d/nginx

# set file permissions
$ sudo chmod +x /etc/init.d/nginx

# add init script run levels
$ sudo /usr/sbin/update-rc.d -f nginx defaults

The last part of this tutorial involves making changes to the nginx conf file: /opt/nginx/conf/nginx.conf

For my server, I set nginx to run as the same user/group as Apache, and increased the number of worker processes (per # of CPU):

# replaced: 
user  nobody;
worker_processes  1;

# with: 
user   www-data www-data;
worker_processes  4;

Within the http directive, I added a server directive for my rails app:

http {
  # ...snip...
  server {
    listen 80;
    server_name rails.eric.vm
    root /var/www/railsdemo/public
    passenger_enabled on;
  }
  # ...snip...
}

And, an upstream and server directive for Apache:

http {
  # ...snip...
  upstream apache {
    server 127.0.0.1:8000 weight=5;
  }

  server {
    listen 80;
    server_name php.eric.vm;

    location / {
      proxy_pass http://apache;
    }
  }
  # ...snip...
}

The above configuration changes allow nginx to listen on port 80, and based on hostname: 1. server the rails app (nginx > passenger > rails); or 2. proxy pass the request to Apache (nginx > apache > php).

* Special thanks to Dan Vine (my Rails partner in crime)!

In this article, I'll share a drush script I wrote to export data from a Drupal site to JSON format. Scripts like this will require customization, but hopefully it will be helpful as a kick start for some. I used it to export users, nodes, comments, taxonomy, and files from a blog site.

<?php

// define data format
DEFINE('EXPORT_DATE_FORMAT', 'Y-m-d H:i:s');

// fetch desired node data
$sql = "select nid from {node} order by nid asc";
$resource = db_query($sql);
$nodes = array();
while($row = db_fetch_object($resource)) {
  $node = node_load($row->nid);
  if (is_object($node)) {
    $nodes[] = $node;
  }
}

// create a container to store all data
$data = new StdClass();

// loop through node objects, collect desired data
$data->nodes = new StdClass();
foreach ($nodes as $nid => $node) {
  
  $n = new StdClass();
  
  // basic properties
  $n->nid = $node->nid;
  $n->type = $node->type;
  $n->uid = $node->uid;
  $n->user_name = $node->name;
  $n->status = $node->status;
  $n->created = date(EXPORT_DATE_FORMAT, $node->created);
  $n->changed = date(EXPORT_DATE_FORMAT, $node->changed);
  $n->title = $node->title;
  $n->body = $node->body;
  $n->path = $node->path;

  // cck field [simple example, single value]
  if (!empty($node->field_example_single[0]['value'])) {
    $n->field_example_single = $node->field_example_single[0]['value'];
  }

  // cck field [simple example, multi-value]
  if (!empty($node->field_example_multi)) {
    $n->field_example_multi = array();
    foreach ($node->field_example_multi as $field_data) {
      $n->field_example_multi[] = $field_data['value'];
    }
  }

  // taxonomy
  if (!empty($node->taxonomy)) {
    $n->taxonomy = array();
    foreach ($node->taxonomy as $tid => $object) {
      $n->taxonomy[] = $object->name;
    }
  }

  // files
  if (!empty($node->files)) {
    $n->files = array();
    foreach ($node->files as $fid => $object) {
      $f = new StdClass();
      $f->fid = $fid;
      $f->filename = $object->filename;
      $f->filepath = $object->filepath;
      $f->filemime = $object->filemime;
      $f->filesize = $object->filesize;
      $f->timestamp = date(EXPORT_DATE_FORMAT, $object->timestamp);
      $n->files[] = $f;
    }

  }

  // comments (recursive)
  if ($node->comment_count) {
    $n->comments = get_node_comments_recursive($n->nid);
  }
  
  // process node type
  if (!isset($data->nodes->{$n->type})) {
    $data->nodes->{$n->type} = array();
  }
  $data->nodes->{$n->type}[$n->nid] = $n;
  
}

// fetch user object list
$sql = "select uid from {users} order by uid asc";
$resource = db_query($sql);
$users = array();
while($row = db_fetch_object($resource)) {
  $user = user_load($row->uid);
  if (is_object($user)) {
    $users[] = $user;
  }
}

// loop through user objects, collect desired data
$data->users = array();
foreach($users as $user) {

  $u = new StdClass();
  
  $u->uid = $user->uid;
  $u->name = $user->name;
  $u->pass = $user->pass;
  $u->email = $user->mail;
  $u->created = date(EXPORT_DATE_FORMAT, $user->created);
  $u->status = $user->status;
  $u->picture = $user->picture;
  $u->roles = array_values($user->roles);
  
  $data->users[$u->uid] = $u;

}

$json = json_encode($data);

file_put_contents('/non/docroot/path/drupal_export.json', $json);

// FUNCTIONS

// recursively fetch comments data
function get_node_comments_recursive($nid, $pid = 0) {
 
  $sql = "
    select *
    from {comments}
    where nid = %d and pid = %d
    order by thread asc
  ";
  $resource = db_query($sql, $nid, $pid);

  $comments = array();
  while ($row = db_fetch_object($resource)) {
    
    $c = new StdClass();
    $c->cid = $row->cid;
    $c->pid = $row->pid;
    $c->nid = $row->nid;
    $c->uid = $row->uid;
    $c->subject = $row->subject;
    $c->comment = $row->comment;
    $c->hostname = $row->hostname;
    $c->timestamp = date(EXPORT_DATE_FORMAT, $row->timestamp);
    $c->status = $row->status;
    $c->thread = $row->thread;
    $c->user_name = $row->name;
    
    $comments[$row->cid] = $c;
  }
  if (empty($comments)) {
    return array();
  }
  
  foreach ($comments as $key => $value) {
    $children = get_node_comments_recursive($nid, $value->cid);
    if (!empty($children)) {
      $comments[$key]->children = $children;
    }
  }
  
  return $comments;
}
?>

I put this script outside my Drupal docroot in a scripts directory. I called it via drush like this:

$ cd drupal_docroot
$ drush scr ../scripts/drupal_export.php

In this quick snippet, I'll show some code that uses the Tika Java library to index the content in WebFM file attachments and add the data as Apache Solr documents. This code is designed to work with the Apache Solr Search Integration Drupal module, and piggyback off of the Apache Solr Attachments module. Out of the box, the Apache Solr Attachments module can index CCK file fields and node file attachments, but the WebFM module uses its own custom file tables and therefore the files are not indexed. This code assumes you have the Tika library already integrated with your Solr installation. Please review the Tika Getting Started documentation for more information.

<?php

/**
 * Implements hook_update_index()
 */
function index_webfm_update_index() {
 
  $cron_limit = 10;
  $rows = apachesolr_get_nodes_to_index('index_webfm', $cron_limit);
  $success = apachesolr_index_nodes($rows, 'index_webfm', '_index_webfm_add_documents');

}

/**
 * Implements custom function to add webfm file attachments content as Solr documents
 */
function _index_webfm_add_documents(&$documents, $nid, $namespace = 'index_webfm') {
  if ($namespace != 'index_webfm') {
    return;
  }
 
  // load node
  $node = node_load($nid);
  if (!is_object($node)) {
    return;
  }
 
  // get webfm fids
  $sql = "
    select a.fid, f.fpath as `filepath`, f.fmime as `filemime`, f.fcreatedate as `timestamp`
    from {webfm_attach} a
    join {webfm_file} f on f.fid = a.fid
    where a.nid = %d
    order by a.weight
  ";
  $resource = db_query($sql, $node->nid);
  $webfm_data = array();
  while ($row = db_fetch_object($resource)) {
    $webfm_data[] = $row;
  }
  
  // ensure webfm files exists for node
  if (!count($webfm_data)) {
    return;
  }
  
  // loop through web fm data, collect data to add to document
  $tika_data = NULL;
  foreach ($webfm_data as $file) {
    
    // direct tika
    if (variable_get('apachesolr_attachment_extract_using', 'tika') == 'tika') {
      
      if (function_exists('apachesolr_attachments_extract_using_tika')) {
        $tika_data = apachesolr_attachments_extract_using_tika($file->filepath);
      }
      
    }
    // tika via solr
    else {
      
      if (function_exists('apachesolr_attachments_extract_using_solr')) {
        list($tika_data, $metadata) = apachesolr_attachments_extract_using_solr($file->filepath);
      }
      
    }
    if (!$tika_data || !is_string($tika_data)) {
      continue;
    }
    
    // create new Solr document
    $document = new Apache_Solr_Document();
    $document->id = apachesolr_document_id($file->fid .'-'. $node->nid, 'file');
    $document->url = file_create_url($file->filepath);
    $document->path = $file->filepath;
    $document->hash = apachesolr_site_hash();
    $document->entity = 'file';
    $document->site = url(NULL, array('absolute' => TRUE));
    $document->nid = $node->nid;
    $document->title = basename($file->filepath);
    $document->created = apachesolr_date_iso($file->timestamp);
    $document->changed = $document->created;
    $document->status = $node->status;
    $document->sticky = $node->sticky;
    $document->promote = $node->promote;
    $document->uid = $node->uid;
    $document->name = $node->name;
    $document->body = apachesolr_clean_text(basename($file->filepath) . ' ' . $tika_data);
    $document->ss_filemime = $file->filemime;
    $document->ss_file_node_title = apachesolr_clean_text($node->title);
    $document->ss_file_node_url = url('node/' . $node->nid, array('absolute' => TRUE));
   
    // add new webfm document to documents
    $documents[] = $document;
        
  } 
}
?>

After implementing this code, when nodes were set to be indexed by Solr, their webfm file attachments were separately processed, the content was extracted from the file attachments, and added as new Solr documents.

In this article, I'll address an issue that has surfaced on every recent project I've been working on: how to update nodes when a CCK field's allowed values list changes. For my projects, I implemented a variety of batch processes and drush scripts to solve this problem. But for this article, I decided to implement a form submit handler solution to automatically update nodes when the CCK field settings page is submitted. Please be careful when implementing code like this, it has potential to update a lot of nodes, and hit memory limitation issues depending on how many nodes need to be updated. As I mentioned before, I implemented a combination of batch APIs and drush scripts to process production environments; I implemented this code in a development environment. Anyway..

I created a new content type "band", and added a CCK text/select field (genre) with an allowed values list:



I used Devel's auto-generate content functionality to create 20 nodes (/admin/generate/content), and then added a simple view to show the node's title and genre:



Now that I had some working test data, I added some code to a custom module to modify the CCK field property settings page, and add a submit handler update the nodes:

<?php

// define module constants
define('CCK_FIELD_PROCESS', 'field_genre');
define('CCK_TYPE_PROCESS', 'band');

/**
 * Implements hook_form_alter()
 */
function helper_form_alter(&$form, &$form_state, $form_id) {
  
  // check for CCK form, CCK type, and CCK field (see above constants)
  if ($form_id == 'content_field_edit_form' && $form['type_name']['#value'] == CCK_TYPE_PROCESS && $form['field_name']['#value'] == CCK_FIELD_PROCESS) {
    
    // get original allowed values
    $original_allowed_values_string = $form['field']['allowed_values_fieldset']['allowed_values']['#default_value'];
    
    // explode values, get array of values    
    $exploded = explode("\n", $original_allowed_values_string);
    $original_allowed_values_array = array();
    foreach ($exploded as $data) {

      // explode data on "|"
      list($key, $value) = $exploded2 = explode('|', $data);      
      $original_allowed_values_array[$key] = $value;
      
    }
    
    // ensure data exists
    if (empty($original_allowed_values_array)) {
      return;
    }

    // store original values in form state storage
    $form_state['storage']['original_allowed_values'] = $original_allowed_values_array;
    
    // store other information in form state
    $form_state['storage']['cck_type'] = CCK_TYPE_PROCESS;
    $form_state['storage']['cck_field'] = CCK_FIELD_PROCESS;

    // add submit handler
    $form['#submit'][] = '_helper_form_alter_content_field_edit_form_submit';
        
  }

}

/**
 * Implements custom form submit handler
 */
function _helper_form_alter_content_field_edit_form_submit($form, &$form_state) {

  // get new allowed values list
  $new_allowed_values_string = $form_state['values']['allowed_values'];
  
  // ensure data exists
  if (empty($new_allowed_values_string)) {
    return;
  }

  // explode values, get array of values    
  $exploded = explode("\n", $new_allowed_values_string);
  $new_allowed_values_array = array();
  foreach ($exploded as $data) {

    // explode data on "|"
    list($key, $value) = $exploded2 = explode('|', $data);      
    $new_allowed_values_array[$key] = $value;
    
  }  

  // ensure data exists
  if (empty($new_allowed_values_array)) {
    return;
  }
  
  // NOTE: the next few lines gets the key values from the allowed values
  // and checks to see which are different.
  // This code assumes any change to the keys (including changing the order) will update the nodes.
  // Update to your heart's content
  $new_keys = array_keys($new_allowed_values_array);
  $old_keys = array_keys($form_state['storage']['original_allowed_values']);
  
  $key_diffs = array();
  foreach ($old_keys as $key => $value) {
    
    if ($new_keys[$key] != $value) {
      $key_diffs[$value] = $new_keys[$key];
    }
    
  }
  
  // ensure data has changed
  if (empty($key_diffs)) {
    unset($form_state['storage']);
    return;
  }
  
  // get data from form storage
  $cck_field = $form_state['storage']['cck_field'];
  $cck_type = $form_state['storage']['cck_type'];
  
  // loop through key diffs
  foreach ($key_diffs as $old => $new) {
  
    // get a list of node ids that need to be updated
    // NOTE: this sql assumes the field is NOT shared
    $sql = "
      select n.nid
      from {node} n
      join {content_type_%s} b on b.nid = n.nid and b.vid = n.vid
      where n.type = '%s' and b.%s_value = '%s'
    ";
    $resource = db_query($sql, $cck_type, $cck_type, $cck_field, $old);
    $node_ids = array();
    while ($row = db_fetch_object($resource)) {
      $node_ids[] = $row->nid;
    }
    
    // ensure results exist
    if (empty($node_ids)) {
      continue;
    }
    
    // loop through node ids
    // NOTE: got a lot of nodes? you might need a batch API instead
    foreach ($node_ids as $nid) {
      
      // load node
      $node = node_load($nid, NULL, TRUE);
      
      // ensure node loaded
      if (!is_object($node)) {
        continue;
      }
      
      // update node properties
      $node->{$cck_field}[0]['value'] = $new;
      $node->revision = TRUE;
      node_save($node);
      
      // remove cck cache for node
      db_query("DELETE FROM {cache_content} WHERE cid LIKE 'content:%d%'", $node->nid);
      
    }
  
  }
  
  // remove form storage
  unset($form_state['storage']);

}
?>

To test this code, I went back to the CCK field settings page (example: /admin/content/node-type/band/fields/field_genre), updated the allowed value list, and saved.



The form submit handler is triggered and the nodes are updated. Refreshing my view shows the updated genre fields on my test nodes:

I've encountered this issue a few times recently when working with Apache Solr (in Drupal 6): clients want to to show all search results and allow the users to browse facets, instead of requiring a search phrase or filter to begin the search. The current version of the Apache Solr module appears to require at least one of these items.

• Querying Solr directly allows you to return all search results using the query/select URL. Example: http://example.com:8080/solr/select


• Entering a keyword works. Example: http://example.com/search/apachesolr_search/blah


• Entering a keyword AND taxonomy filter works. Example: http://example.com/search/apachesolr_search/blah?filters=tid%3A22


• Entering a taxonomy filter WITHOUT a keyword works. Example: http://example.com/search/apachesolr_search/?filters=tid%3A22


• No keyword AND no filter returns nothing (dang). Example: http://example.com/search/apachesolr_search

A quick look at apachesolr_search.module, in the function: apachesolr_search_view() shows this is not going to happen:

<?php
$keys = trim(search_get_keys());
// ...snip...
$filters = trim($_GET['filters']);
// ...snip...
if ($keys || $filters) {
  // do search
}
?>

So, what if there was a way to modify the $_GET values prior to the apache solr search page callback? hook_boot maybe? NOTE: hook_boot functions are only processed if the bootstrap column in the system table for your module is set to 1! Example SQL to update..

update {system}
set bootstrap = 1
where type = 'module' and name = 'MYMODULE'

I then created this hook_boot function in my module. It is run prior to the the apache solr search page callback, and allows me to tamper with $_GET variables. In this case, I just add a fake filter to allow the search to pass the above conditional statements.

<?php
function MYMODULE_boot() {
  
  // check for search page
  $search_path = 'search/apachesolr_search';
  if (substr($_GET['q'], 0, strlen($search_path))==$search_path) {
    
    // get search phrase
    $search_phrase = substr($_GET['q'], strlen($search_path)+1);
    
    // get filters
    $filters = $_GET['filters'];
    
    // check if filters AND search phrase do not exist
    if (empty($filters) && empty($search_phrase)) {
          
      // set fake filter
      $_GET['filters'] = 1;
      
    }
  }
  
}
?>

Now, going to the solr search page defaults to show all results and facets:



It appears to work, but anyone know of a better way to accomplish this?! :)