email | Eric London's Blog

Setting up a script to ensure the Apache Solr java process is running

Submitted by Eric.London on 4/10/2011

Tags:

2 comments

I recently found some time to switch my site's search framework from Lucene to Apache Solr. The module's README.txt makes installation for small production sites easy and straight forward.

Following the installation guide, I started the java Solr process by entering the right directory and executing the java jar..

$ java -jar start.jar

Everything was up and running in minutes.. until I closed my terminal and the java service ended with my shell process. Short term, I decided to writing a bash shell script to ensure Solr is running, and cron it to run every five minutes.

Here are the contents of my bash shell script:

#!/bin/bash

# check for process id
pid=`ps ax | grep -ir java.*jar.*start\.jar | grep -iv grep | awk '{print $1}'`

# check if pid is not an integer
if ! [[ "$pid" =~ ^[0-9]+$ ]] ; then

  # start service
  cd /path/to/my/apache-solr-1.4.1/installation
  java -jar start.jar &

  # send email notification
  message='ericlondon.com: starting solr service'
  subject='ericlondon.com: starting solr service'
  to='myemail@example.com'
  echo "$message" | mail -s "$subject" $to

  exit 1;

fi

And I added the following cronjob:

$ crontab -l
*/5 * * * * /path/to/my/scripts/folder/check_solr.sh

A better option would be to setup initialization scripts for the process (/etc/init.d/), or install Solr as a more permanent solution, but I guess this will do for the time being :) ...

Part 2, Using Supervisor (updated: 2011/04/12)

As mentioned above, using a cronjob is probably not the best solution. I decided to install and configure supervisord to monitor the process.

Unfortunately supervisor was not available for for Centos 5.5 (RHEL):

$ yum search supervisor
Finished
Warning: No matches found for: supervisor
No Matches found

Luckily, I found some RPMs via http://rpmfind.net. I installed supervisor and its one dependency:

# downloading RPMs
$ wget ftp://rpmfind.net/linux/epel/5/x86_64/supervisor-2.1-3.el5.noarch.rpm
$ wget ftp://rpmfind.net/linux/epel/5/x86_64/python-meld3-0.6.3-1.el5.x86_64.rpm

# installing RPMs
$ rpm -Uvh python-meld3-0.6.3-1.el5.x86_64.rpm
$ rpm -Uvh supervisor-2.1-3.el5.noarch.rpm

# setting run level for supervisord
$ chkconfig --level 2345 supervisord on

# starting supervisor
$ /etc/init.d/supervisord start

Next, I create a simple shell script to start the Solr process and made the script executable. NOTE: file contents have been simplified:

#!/bin/bash

# enter solr dir
cd /path/to/my/apache-solr-1.4.1/installation

# start solr
java -jar start.jar

Lastly, I added a few line to my supervisor conf file (/etc/supervisord.conf):

[program:apache_solr]
command=/path/to/my/scripts/folder/apache-solr-supervisor-run.sh

Upon restarting supervisor, solr started automatically

$ /etc/init.d/supervisord restart

$ ps aux | grep -ir java | grep -iv grep
root     28670  0.1  8.5 1041076 43548 ?       Sl   13:30   0:02 java -jar start.jar

I killed the script and it immediately came back (with a different process ID)!

$ kill 28670

$ ps aux | grep -ir java | grep -iv grep
root     28869 62.0  5.3 1021532 27016 ?       Sl   13:50   0:00 java -jar start.jar

Creating a module to email severe log entries to users

Submitted by Eric.London on 10/2/2010

Tags:

Add new comment

In this code snippet, I'll explain how you can create a module that emails log entries of a defined severity to a list of users.

The first step is to create an admin setting page that allows the admin to define the email recipients.

<?php
/**
 * Implements hook_perm()
 */
function logemail_perm() {
  return array(
    'administer log email',
  );
}

/**
 * Implements hook_menu()
 */
function logemail_menu() {

  $items = array();

  // admin settings page callback
  $items['admin/settings/log-email'] = array(
    'title' => t('Log Email'),
    'description' => t('Configure Log Email settings'),
    'page callback' => 'drupal_get_form',
    'page arguments' => array('logemail_callback_admin_settings'),
    'access arguments' => array('administer log email'),
  );
  
  return $items;
}

/**
 * Implements form page callback for module admin settings page
 */
function logemail_callback_admin_settings() {
  
  $form = array();
  
  $form['logemail_recipients'] = array(
    '#type' => 'textfield',
    '#title' => t('Email Recipients'),
    '#default_value' => variable_get('logemail_recipients', ''),
    '#description' => t('Enter email addresses comma separated.'),
    '#required' => TRUE,
  );
  
  return system_settings_form($form);
}

/**
 * Implements validation callback for admin settings form
 */
function logemail_callback_admin_settings_validate($form, &$form_state) {
  
  // explode email addresses on comma
  $emails = explode(',', trim($form_state['values']['logemail_recipients']));
  
  if (!count($emails) || !is_array($emails)) {
    form_set_error('logemail_recipients', t('Invalid email addresses.'));
    return;
  }
  
  // loop through each email address and validate
  foreach ($emails as $key => $value) {

    // validate email address
    $validate_result = valid_email_address(trim($value));

    if ($validate_result !== 1) {
      form_set_error('logemail_recipients', t('Invalid email addresses.'));
      return;
    }
    else {
      // trim email addresses
      $emails[$key] = trim($value);
    }
  }
  
  // set trimmed values
  $form_state['values']['logemail_recipients'] = implode(',', $emails);
}
?>

The above code creates an admin setting page that allows the admin to define the recipients list, and implements validation to ensure the email addresses exist and are valid.

Log Email Admin Settings

The next two functions implement hook_mail() and hook_watchdog() to check the severity of the log entry, compose the email message, and send it.

<?php
/**
 * Implements hook_watchdog()
 */
function logemail_watchdog($log_entry) {
  
  // check severity of log entry
  if ($log_entry['severity']<5) {

    // define parameters for drupal_mail
    $module = 'logemail';
    $key = 'watchdog';
    $to = variable_get('logemail_recipients', '');
    $language = language_default();
    $params = array(
      'log_entry' => $log_entry,
    );

    // send email using drupal_mail
    drupal_mail($module, $key, $to, $language, $params);
  }
}

/**
 * Implements hook_mail()
 */
function logemail_mail($key, &$message, $params) {
  switch ($key) {
    case 'watchdog':
      // set email subject
      $message['subject'] = t("Log Email Watchdog") . ": " . $params['log_entry']['type'];

      // define email body - start
      
      // process variables using t()
      $email_body = t($params['log_entry']['message'], $params['log_entry']['variables']);

      // decode html special characters
      $email_body = htmlspecialchars_decode($email_body);

      // remove html tags
      $email_body = strip_tags($email_body);
      
      // define email body - end
      
      // set email body
      $message['body'] = $email_body;
      
      break;
  }
}
?>

Now, when log entries are created that match the defined severity, they are emailed to the recipient list.

Configuring a server to parse email via a PHP script

Submitted by Eric.London on 1/10/2010

Tags:

4 comments

In this tutorial I'll show how you can setup a server to parse email with a PHP script. This tutorial assumes that your server is configured to receive email (I wrote this using a virtual machine running postfix).

The first thing you'll need to do is configure an alias to direct email to a PHP script (instead of an email box). I added the following entry to the bottom of my /etc/aliases file and then ran the "newaliases" command to refresh my aliases database:

phpscript: "|php -q /usr/local/bin/email.php"

The above entry will pipe email sent to phpscript@MYDOMAIN to the designated PHP script.

And here's the script:

#!/usr/bin/php
<?php

// fetch data from stdin
$data = file_get_contents("php://stdin");

// extract the body
// NOTE: a properly formatted email's first empty line defines the separation between the headers and the message body
list($data, $body) = explode("\n\n", $data, 2);

// explode on new line
$data = explode("\n", $data);

// define a variable map of known headers
$patterns = array(
  'Return-Path',
  'X-Original-To',
  'Delivered-To',
  'Received',
  'To',
  'Message-Id',
  'Date',
  'From',
  'Subject',
);

// define a variable to hold parsed headers
$headers = array();

// loop through data
foreach ($data as $data_line) {

  // for each line, assume a match does not exist yet
  $pattern_match_exists = false;

  // check for lines that start with white space
  // NOTE: if a line starts with a white space, it signifies a continuation of the previous header
  if ((substr($data_line,0,1)==' ' || substr($data_line,0,1)=="\t") && $last_match) {

    // append to last header
    $headers[$last_match][] = $data_line;
    continue;

  }

  // loop through patterns
  foreach ($patterns as $key => $pattern) {

    // create preg regex
    $preg_pattern = '/^' . $pattern .': (.*)$/';

    // execute preg
    preg_match($preg_pattern, $data_line, $matches);

    // check if preg matches exist
    if (count($matches)) { 

      $headers[$pattern][] = $matches[1];
      $pattern_match_exists = true;
      $last_match = $pattern;

    }

  }

  // check if a pattern did not match for this line
  if (!$pattern_match_exists) {
    $headers['UNMATCHED'][] = $data_line;
  }

}

?>

At this point in the code, the body of the message will be contained in the $body variable and the headers will be in $headers.

Here is an example of the parsed headers (using print_r()):

Array
(
    [UNMATCHED] => Array
        (
            [0] => From root@Eric-Centos.localdomain  Sun Jan 10 21:49:50 2010
        )

    [Return-Path] => Array
        (
            [0] => <root@Eric-Centos.localdomain>
        )

    [X-Original-To] => Array
        (
            [0] => phpscript
        )

    [Delivered-To] => Array
        (
            [0] => phpscript@Eric-Centos.localdomain
        )

    [Received] => Array
        (
            [0] => by Eric-Centos.localdomain (Postfix, from userid 0)
            [1] => 	id 4D03F30131; Sun, 10 Jan 2010 21:49:50 -0500 (EST)
        )

    [To] => Array
        (
            [0] => phpscript@Eric-Centos.localdomain
        )

    [Subject] => Array
        (
            [0] => This is the subject
        )

    [Message-Id] => Array
        (
            [0] => <20100111024950.4D03F30131@Eric-Centos.localdomain>
        )

    [Date] => Array
        (
            [0] => Sun, 10 Jan 2010 21:49:50 -0500 (EST)
        )

    [From] => Array
        (
            [0] => root@Eric-Centos.localdomain (root)
        )

)

Now, you have all the email headers and message body parsed. You can do whatever your heart desires with the data, like insert it into a database or even create nodes!

Removing a column from a view for unauthenticated users using hook_views_pre_build()

Submitted by Eric.London on 10/12/2009

Tags:

10 comments

In this tutorial I'll show how you can programmatically and dynamically remove a column from your view. In my example, I chose to hide an email column from unauthenticated users, but you could apply this code to do pretty much anything.

To get things started, I defined a content type of "Profile" to contain some user details (using the Content Profile module) and used the Devel module's generate users and nodes functionality to create some sample data. I then created a view to show the data:

Here is my page view display:

Now you can add the hook_views_pre_build() function in your module. Since the $view object is large, I recommend using krumo() (which comes with the Devel module) to browse through the $view object's properties.

<?php
function MYMODULE_views_pre_build(&$view) {

  // check for your view name (in my example: people)
  if ($view->name=='people') {
    krumo($view);
  }

}
?>

The above code will neatly format the $view object in a hierarchical display. You can click to expand each class property.

As you can see, the $view object has a lot of data in it. At this point, you'll need to get acquainted with the structure of the handler object of the view ($view->display['default']->handler) and determine what to modify. To remove the email column for unauthenticated users, you can add the following code:

<?php
function MYMODULE_views_pre_build(&$view) {

  // check for your view name (in my example: people)
  if ($view->name=='people') {
   
    // check if the user is anon
    if (in_array('anonymous user',$GLOBALS['user']->roles)) {

      // remove view hander properties for email column
      unset(
        $view->display['default']->handler->options['fields']['mail'],
        $view->display['default']->handler->options['style_options']['columns']['mail'],
        $view->display['default']->handler->options['style_options']['info']['mail']
      );

    }

  }

}
?>

When I logout, my view no longer shows the email column:

Uploading a file via the Forms API and attaching it to an email

Submitted by Eric.London on 4/24/2009

Tags:

4 comments

In this tutorial I'll show how I used the Forms API to add a file upload field and attach the uploaded file in an email. I have experience using the PEAR libraries Mail and Mail_MIME to handle MIME/HTML emails and file attachments, so I decided to use them.

If you are unfamiliar with PEAR, here are a few quick tips:

# install pear via YUM (this will vary across operating system)
$ sudo yum install php-pear

# upgrade all PEAR packages
$ sudo yum upgrade-all

# check for Mail and Mail_MIME libraries
$ pear list | grep -i mail
Mail             1.1.14  stable
Mail_Mime        1.5.2   stable

# install a PEAR library
$ sudo pear install Mail

Now we can create our form callback function. NOTE: I'm keeping this form as simple as possible to focus on the file attachment functionality.

<?php
function _MYMODULE_form() {
  // create an empty form array
  $form = array();

  // set the form encoding type
  $form['#attributes']['enctype'] = "multipart/form-data";

  // add a file upload file
  $form['upload'] = array(
    '#type' => 'file',
    '#title' => t('Attach a file'),
  );
    
  // add a submit button
  $form['submit'] = array(
    '#type' => 'submit',
    '#value' => 'Submit',
  );

}
?>

The above will create a basic form object with a file upload file and a submit button. The form can be included using the drupal_get_form function. Example:

<?php
$html = drupal_get_form('_MYMODULE_form');
?>

Next I added a validation function to validate the file upload.

<?php
function _MYMODULE_form_validate($form, &$form_state) {

  // define upload field name
  // NOTE: this should match the name of your form file field
  $fieldName = 'upload';
    
  // If a file was uploaded, process it.
  if (isset($_FILES['files']) && is_uploaded_file($_FILES['files']['tmp_name'][$fieldName])) {

    // attempt to save the uploaded file
    $file = file_save_upload($fieldName);

    // set error if file was not uploaded
    if (!$file) {
      form_set_error($fieldName, 'Error uploading file.');
      return;
    }
        
    // set files to form_state, to process when form is submitted 
    $form_state['values']['file'] = $file;
        
  }
  else {
    // set error
    form_set_error($fieldName, 'Error uploading file.');
    return;    
  }
        
}
?>

The above validation form will check to see if a file has been uploaded and set a form error as necessary. The last line of the function sets information about the successfully uploaded file to the $form_state array, which will then be passed to the submit handler function.

Next I created a form submit handler function which will create a Mail_Mime object, attach the file, send the email, and set a drupal message for the user to see.

<?php
function _MYMODULE_form_submit($form, &$form_state) {

  // create the email subject
  $subject = 'File attachment form submitted';
    
  // create the text version of the email body
  $body_text = "Dear Eric,\n\nblah blah blah.\nblah blah blah.\n\nRegards,\nEric";
    
  // create an html version of the email
  $body_html = str_replace("\n", "<br>", $body_text);

  // define who receives the email
  $to = "YOUREMAILADDRESS";

  // include pear libraries
  require_once('Mail.php');
  require_once('Mail/mime.php');
    
  // create new mail mime object
  $mime = new Mail_mime("\n");

  // add attachment
  if ($form_state['values']['file']) {
    $mime->addAttachment(
      $form_state['values']['file']->filepath,
      $form_state['values']['file']->filemime,
      $form_state['values']['file']->filename
    );
  }
    
  // set text message
  $mime->setTXTBody($body_text);
    
  // set html message
  $mime->setHTMLBody($body_html);
    
  // get message body
  $body = $mime->get();

  // define headers
  $hdrs = array(
    'From' => variable_get('site_mail','YOUREMAILADDRESS'),
    'Subject' => $subject,
  );
    
  // process headers
  $hdrs = $mime->headers($hdrs);
    
  // create mail object
  $mail =& Mail::factory('mail');

  // send email
  $mail->send($to, $hdrs, $body);   

  // set message to user
  drupal_set_message('The file attachment form has been submitted.');    

}
?>

If everything worked correctly, the form will be validated, submitted, and an email with the file attachment will be sent.