background image
HomeRecent PostsDrupalSearchTagsRSSContactAboutAccount

Programmatically generate all images for each imagecache preset using a Drush PHP script
Eric.London's picture
Submitted by Eric.London on 11/26/2010
Tags:
  • 3 comments
  • Share this

My Drupal photo gallery implements a total of 9 imagecache presets, but due to a PHP memory_limit cap of 90MB in my previous hosting contract, I frequently encountered issues generating images.

Since I was on a shared hosting plan, PHP was executed as CGI and a service ran on the server to kill PHP scripts that reached the PHP memory_limit. This resulted in the following error, which prevented imagecache from creating missing images and redirecting the user properly.

Premature end of script headers: index.php

After changing my hosting plan and increasing my server resources, I decided to write a PHP script to programmatically generate all the missing images. This script when executed on the shell using Drush, will get a list of every image in the files directory, and make an HTTP request for each imagecache preset URL.

<?php
// store original working directory
define('ORIGINAL_DIRECTORY', getcwd());

// define site http host
define('HTTP_HOST','pics.ericlondon.com');

// get imagecache presets data
$presets_data = imagecache_presets();

// loop through presets data and collect preset names
$presets = array();
foreach ($presets_data as $key => $value) {
  $presets[] = $value['presetname'];
}

// get a list of pictures
chdir(file_directory_path());
// NOTE: the following line uses find, grep, and sed to generate a list of image files.
// It will vary depending on which file extensions to include and which directories to ignore
$command = 'find . -type f | egrep -ir "\.(gif|jpeg|jpg|png)$" | sed "s/^\.\///" | egrep -iv "^(imagecache|imagefield_thumbs)\/"';
$output = `$command`;
$files = explode("\n", trim($output));
chdir(ORIGINAL_DIRECTORY);

// loop through file list
foreach ($files as $file) {

  // loop through each preset
  foreach ($presets as $preset) {
 
    // define url
    $url = 'http://' . HTTP_HOST . '/' . file_directory_path() . '/imagecache/' . $preset . '/' . $file;
   
    // request url
    $http_request_data = drupal_http_request($url);
   
    // log entry
    $log_entry = $http_request_data->code . " " . $http_request_data->status_message . " " . $url;
    file_put_contents('ic_preset_log.txt', $log_entry . "\n", FILE_APPEND);
 
  }

}
?>

I executed this script on the shell using the following drush command:

drush scr generate_presets.php

The script logs every request and can be reviewed afterward..

$ tail ic_preset_log.txt
200 OK http://pics.ericlondon.com/sites/ericlondon.com/files/imagecache/large_l...
200 OK http://pics.ericlondon.com/sites/ericlondon.com/files/imagecache/large_p...
200 OK http://pics.ericlondon.com/sites/ericlondon.com/files/imagecache/medium/...
200 OK http://pics.ericlondon.com/sites/ericlondon.com/files/imagecache/medium_...
200 OK http://pics.ericlondon.com/sites/ericlondon.com/files/imagecache/medium_...
200 OK http://pics.ericlondon.com/sites/ericlondon.com/files/imagecache/thumbna...
200 OK http://pics.ericlondon.com/sites/ericlondon.com/files/imagecache/thumbna...
200 OK http://pics.ericlondon.com/sites/ericlondon.com/files/imagecache/thumbna...
200 OK http://pics.ericlondon.com/sites/ericlondon.com/files/imagecache/large/u...
200 OK http://pics.ericlondon.com/sites/ericlondon.com/files/imagecache/large_landscape/userpics/10020/2007_07_21_003.JPG

Creating a Batch API operation to parse a large CSV file
Eric.London's picture
Submitted by Eric.London on 1/6/2010
Tags:

In this code snippet, I'll show how you can parse a (large) CSV file using Drupal's Batch API. The purpose of batching an operation is to avoid PHP memory limits and time outs. Before you begin, I recommend reviewing the following two articles. Be sure to review the additional batch parameters outlined in the documentation, you might need to use them.

Batch API
Batch Operations

We can start by defining the arguments that will be passed into the batch_set() function. For this example, I added this code in an arbitrary page callback function.

<?php
function MYMODULE_callback_csv_import() {

  // define path to CSV file
  $csv_file_path = file_directory_path() . '/import_path/myfile.csv';

  // define a redirect path upon batch completion
  $redirect_path = 'admin/import-csv';

  // define batch array structure
  // NOTE: minimal parameters defined to simplify code
  $batch = array(
    'title' => t('Reading File'),
    'operations' => array(
      array(
        '_MYMODULE_batch_read', array($csv_file_path),
      ),
    ),
  );

  // set batch
  batch_set($batch);

  // process batch
  batch_process($redirect_path);

}
?>

Next, we'll define the batch callback function. This function will be called repeatedly until the $context['finished'] variable is set to "1".

<?php
function _MYMODULE_batch_read($csv_file_path, &$context) {
 
  // define batch limit
  $batch_limit = 100;

  // assume the batch process has not completed
  $context['finished'] = 0;

  // open the file for reading
  $file_handle = fopen($csv_file_path, 'r');

  // check if file pointer position exists in the sandbox, and jump to location in file
  if ($context['sandbox']['file_pointer_position']) {
    fseek($file_handle, $context['sandbox']['file_pointer_position']);
  }
 
  // loop through the file and stop at batch limit
  for ($i = 0; $i < $batch_limit; $i++) {

    // get file line as csv
    $csv_line = fgetcsv($file_handle);

    // NOTE: at this point, do what ever you'd like with the CSV array data!
    if (is_array($csv_line)) {
      // db_query(), etc   
    }

    // retain current file pointer position
    $context['sandbox']['file_pointer_position'] = ftell($file_handle);

    // check for EOF
    if (feof($file_handle)) {
      // complete the batch process
      $context['finished'] = 1;

      // end loop
      break;
    }

  }

}
?>

The batch operation will be called until the end of the CSV file is reached. The $context variable is passed by reference into the batch callback so you can maintain data through each iteration; in this case, the position of the file pointer. When the batch operation is complete, the user will be redirected to the batch_process() path argument.

It's important to read the full Batch API documentation so you can take advantage of its additional features: finished callback, init_message, progress_message, error_message, etc.

Syndicate content