This blog article in an update to a previous post Geospatial Apache Solr searching in Drupal 7 using the Search API module (Ubuntu version). In this article, I will outline a step by step guide on how to integrate Drupal 7 with the Search API modules to implement GeoSpatial searching in a View. I used a clean Ubuntu server (12.04 LTS) installation on a VirtualBox virtual machine.

Initial package installation

# update system packages
apt-get update
apt-get upgrade -y

# install ssh server
apt-get install openssh-server -y

# install apache + php
apt-get install php5 php5-cli php5-curl php5-common php5-gd php5-mysql php-pear -y

# install mysql server
apt-get install mysql-server -y

# install tomcat/java
apt-get install tomcat7 tomcat7-admin openjdk-7-jdk -y

# install git
apt-get install git

# install drush via pear
pear channel-discover pear.drush.org
pear install drush/drush
pear upgrade-all

Setting up MySQL

# add mysql db and user
mysql -p --execute="create database drupal7"
mysql -p --execute="GRANT ALL PRIVILEGES ON drupal7.* TO 'drupal7'@'localhost' IDENTIFIED BY 'lapurd'"

Creating the Apache vhost

# create directory
mkdir -p /var/www/vhosts/drupal7
cd /var/www/vhosts/drupal7

# download drupal via drush
drush dl drupal-7
mv drupal-7.15 html

# create a new git repo
git init
git add .
git commit -am "initial commit"

# install drupal via drush
cd html
drush site-install minimal --db-url=mysql://drupal7:lapurd@localhost/drupal7 --account-mail=drupal@ericlondon.com --account-pass=lapurd

Apache Configuration

# add apache vhost
# new file: /etc/apache2/sites-available/drupal7
<VirtualHost *:80>
	ServerAdmin webmaster@drupal7.dev

	ServerName drupal7.dev

	DocumentRoot /var/www/vhosts/drupal7/html
	<Directory /var/www/vhosts/drupal7/html>
		AllowOverride All
	</Directory>

	ErrorLog ${APACHE_LOG_DIR}/drupal7-error.log

	# Possible values include: debug, info, notice, warn, error, crit, alert, emerg.
	LogLevel warn

	CustomLog ${APACHE_LOG_DIR}/drupal7-access.log combined

</VirtualHost>

# enable apache vhost
cd /etc/apache2/sites-enabled
rm 000-default
ln -s ../sites-available/drupal7 000-drupal7

# enable apache mod_rewrite
a2enmod rewrite

# restart apache
apache2ctl restart

Drupal modules setup

cd /var/www/vhosts/drupal7/html

# core modules
drush en field_ui number options taxonomy list -y

# admin menu
drush dl admin_menu -y
drush en admin_menu -y

# views dependency
drush dl ctools -y
drush en ctools -y

# views
drush dl views -y
drush en views views_ui -y

# entity, search api dependency
drush dl entity -y
drush en entity -y

# facetapi, search api dependency
drush dl facetapi -y
drush en facetapi -y

# search api
drush dl search_api -y
drush en search_api search_api_views search_api_facetapi -y

# search_api_solr
mkdir sites/all/libraries
cd sites/all/libraries
wget http://solr-php-client.googlecode.com/files/SolrPhpClient.r60.2011-05-04.tgz
tar -xzf SolrPhpClient.r60.2011-05-04.tgz
cd ../../..
drush dl search_api_solr -y
drush en search_api_solr -y

# geofield module
drush dl geofield -y
drush en geofield -y

Tomcat & Solr setup

# edit file: /etc/tomcat7/tomcat-users.xml
diff tomcat-users.xml tomcat-users.xml.bak.orig 
29,33c29,35
< 
<   <role rolename="admin"/>
<   <role rolename="manager"/>
<   <user username="eric" password="lapurd" roles="admin,manager"/>
< 
---
> <!--
>   <role rolename="tomcat"/>
>   <role rolename="role1"/>
>   <user username="tomcat" password="tomcat" roles="tomcat"/>
>   <user username="both" password="tomcat" roles="tomcat,role1"/>
>   <user username="role1" password="tomcat" roles="role1"/>
> -->

# Download Solr & copy files
mkdir ~/downloads
cd ~/downloads
wget http://archive.apache.org/dist/lucene/solr/3.5.0/apache-solr-3.5.0.tgz
tar -xzf apache-solr-3.5.0.tgz
cp ~/downloads/apache-solr-3.5.0/dist/apache-solr-3.5.0.war /var/lib/tomcat7/webapps/solr.war
cp -r ~/downloads/apache-solr-3.5.0/example/solr/ /var/lib/tomcat7/solr/

# new file: /etc/tomcat7/Catalina/localhost
<?xml version="1.0" encoding="UTF-8"?>
<Context docBase="/var/lib/tomcat7/webapps/solr.war" debug="0" privileged="true" allowLinking="true" crossContext="true">
<Environment name="solr/home" type="java.lang.String" value="/var/lib/tomcat7/solr" override="true" />
</Context>

# edit file: /var/lib/tomcat7/webapps/solr/WEB-INF/web.xml
Uncomment & change this line:
    <env-entry-value>/put/your/solr/home/here</env-entry-value>
to:
    <env-entry-value>/var/lib/tomcat7/solr</env-entry-value>

# copy files from search_api_solr module to tomcat directory
cp sites/all/modules/search_api_solr/schema.xml /var/lib/tomcat7/solr/conf/
cp sites/all/modules/search_api_solr/solrconfig.xml /var/lib/tomcat7/solr/conf/

# reset tomcat permissions
chown -R tomcat7.tomcat7 /var/lib/tomcat7/solr

# restart tomcat
service tomcat7 restart

# NOTE: admin interface URL
http://localhost:8080/solr/admin/

Drupal configuration

Add content type
- URL: /admin/structure/types/add
  - Name: Place (for example)
  - Save content type

Manage fields on new content type
- URL: /admin/structure/types/manage/place/fields
  - Add new field
    - Label: Geo
    - Machine name: field_geo
    - Field Type: Geofield
    - Widget: Latitude / Longitude
  - Save

Add Solr Server
- URL: /admin/config/search/search_api
- Add Server
  - Enter Server Name
  - Service class: choose Solr service
    - Enter Solr host (localhost), port (8080), path (/solr)
  - Save settings

Add solr index
- URL: /admin/config/search/search_api/add_index
  - Index name: GeoSolr
  - Item type: Node
  - Server: 
- Create index

Select fields to index
- URL: /admin/config/search/search_api/index/geosolr/fields
  - Add related fields
    - Choose "Geo", Add fields
    - Choose "The main body text", Add fields
  - Node ID, Integer
  - Title, Fulltext
  - Geo » LatLong Pair » String
  - The main body text » Text
- Save Changes

Added content
- URL: /node/add/place
  - Added coordinates manually using: http://gmaps-samples.googlecode.com/svn/trunk/geocoder/singlegeocode.html/

Index content
- URL: /admin/config/search/search_api/index/geosolr/status

NOTE: Execute direct Solr query to verify data and fields:
http://localhost:8080/solr/select/?q=&fl=*

Solr configuration changes per GeoSpatial

# changes to schema.xml:
root@ubuntu:/var/lib/tomcat7/solr/conf# diff schema.xml schema.xml.bak.orig 
288,291d287
<     <fieldType name="point" class="solr.PointType" dimension="2" subFieldSuffix="_d"/>
<     <fieldType name="location" class="solr.LatLonType" subFieldSuffix="_coordinate"/>
<     <fieldtype name="geohash" class="solr.GeoHashField"/>
< 
296,299d291
< 
<    <field name="ss_field_geo:latlon" type="location" indexed="true" stored="true"/>
<    <dynamicField name="*_coordinate"  type="tdouble" indexed="true"  stored="false"/>
<

# restart tomcat
service tomcat7 restart

# direct solr query to verify geospatial results (using coordinates from Nashua, NH)
http://localhost:8080/solr/select/?q=&fl=*&fq={!geofilt%20sfield=ss_field_geo:latlon%20pt=42.933692,-72.278141%20d=100}

Add new Drupal view

- URL: /admin/structure/views/add
  - view name: GeoSearch
  - show GeoSolr sorted by Unsorted
  - Create a page
    - path: geosearch
  - save and continue
  - Format: Unformatted list
  - Show: Fields
    - Indexed Node: Node ID (Node ID)
    - Indexed Node: Geo (Geo)
    - Indexed Node: Title (Title)
    - The main body text: Text (indexed) (Text (indexed))
  - Save

Lastly, I added a new module "geosearch" and added a hook_search_api_solr_query_alter() function to add the coordinates. It's up to you how you want to integrate the GeoSpatial integration with your Views and site.

function geosearch_search_api_solr_query_alter(array &$call_args, SearchApiQueryInterface $query) {

  $lat = 42.933692;
  $lng = -72.278141;
  $distance = 100;

  $call_args['params']['fq'][] = "{!geofilt sfield=ss_field_geo:latlon pt={$lat},{$lng} d={$distance}}";

}

In this article, I'll share a drush script I wrote to export data from a Drupal site to JSON format. Scripts like this will require customization, but hopefully it will be helpful as a kick start for some. I used it to export users, nodes, comments, taxonomy, and files from a blog site.

<?php

// define data format
DEFINE('EXPORT_DATE_FORMAT', 'Y-m-d H:i:s');

// fetch desired node data
$sql = "select nid from {node} order by nid asc";
$resource = db_query($sql);
$nodes = array();
while($row = db_fetch_object($resource)) {
  $node = node_load($row->nid);
  if (is_object($node)) {
    $nodes[] = $node;
  }
}

// create a container to store all data
$data = new StdClass();

// loop through node objects, collect desired data
$data->nodes = new StdClass();
foreach ($nodes as $nid => $node) {
  
  $n = new StdClass();
  
  // basic properties
  $n->nid = $node->nid;
  $n->type = $node->type;
  $n->uid = $node->uid;
  $n->user_name = $node->name;
  $n->status = $node->status;
  $n->created = date(EXPORT_DATE_FORMAT, $node->created);
  $n->changed = date(EXPORT_DATE_FORMAT, $node->changed);
  $n->title = $node->title;
  $n->body = $node->body;
  $n->path = $node->path;

  // cck field [simple example, single value]
  if (!empty($node->field_example_single[0]['value'])) {
    $n->field_example_single = $node->field_example_single[0]['value'];
  }

  // cck field [simple example, multi-value]
  if (!empty($node->field_example_multi)) {
    $n->field_example_multi = array();
    foreach ($node->field_example_multi as $field_data) {
      $n->field_example_multi[] = $field_data['value'];
    }
  }

  // taxonomy
  if (!empty($node->taxonomy)) {
    $n->taxonomy = array();
    foreach ($node->taxonomy as $tid => $object) {
      $n->taxonomy[] = $object->name;
    }
  }

  // files
  if (!empty($node->files)) {
    $n->files = array();
    foreach ($node->files as $fid => $object) {
      $f = new StdClass();
      $f->fid = $fid;
      $f->filename = $object->filename;
      $f->filepath = $object->filepath;
      $f->filemime = $object->filemime;
      $f->filesize = $object->filesize;
      $f->timestamp = date(EXPORT_DATE_FORMAT, $object->timestamp);
      $n->files[] = $f;
    }

  }

  // comments (recursive)
  if ($node->comment_count) {
    $n->comments = get_node_comments_recursive($n->nid);
  }
  
  // process node type
  if (!isset($data->nodes->{$n->type})) {
    $data->nodes->{$n->type} = array();
  }
  $data->nodes->{$n->type}[$n->nid] = $n;
  
}

// fetch user object list
$sql = "select uid from {users} order by uid asc";
$resource = db_query($sql);
$users = array();
while($row = db_fetch_object($resource)) {
  $user = user_load($row->uid);
  if (is_object($user)) {
    $users[] = $user;
  }
}

// loop through user objects, collect desired data
$data->users = array();
foreach($users as $user) {

  $u = new StdClass();
  
  $u->uid = $user->uid;
  $u->name = $user->name;
  $u->pass = $user->pass;
  $u->email = $user->mail;
  $u->created = date(EXPORT_DATE_FORMAT, $user->created);
  $u->status = $user->status;
  $u->picture = $user->picture;
  $u->roles = array_values($user->roles);
  
  $data->users[$u->uid] = $u;

}

$json = json_encode($data);

file_put_contents('/non/docroot/path/drupal_export.json', $json);

// FUNCTIONS

// recursively fetch comments data
function get_node_comments_recursive($nid, $pid = 0) {
 
  $sql = "
    select *
    from {comments}
    where nid = %d and pid = %d
    order by thread asc
  ";
  $resource = db_query($sql, $nid, $pid);

  $comments = array();
  while ($row = db_fetch_object($resource)) {
    
    $c = new StdClass();
    $c->cid = $row->cid;
    $c->pid = $row->pid;
    $c->nid = $row->nid;
    $c->uid = $row->uid;
    $c->subject = $row->subject;
    $c->comment = $row->comment;
    $c->hostname = $row->hostname;
    $c->timestamp = date(EXPORT_DATE_FORMAT, $row->timestamp);
    $c->status = $row->status;
    $c->thread = $row->thread;
    $c->user_name = $row->name;
    
    $comments[$row->cid] = $c;
  }
  if (empty($comments)) {
    return array();
  }
  
  foreach ($comments as $key => $value) {
    $children = get_node_comments_recursive($nid, $value->cid);
    if (!empty($children)) {
      $comments[$key]->children = $children;
    }
  }
  
  return $comments;
}
?>

I put this script outside my Drupal docroot in a scripts directory. I called it via drush like this:

$ cd drupal_docroot
$ drush scr ../scripts/drupal_export.php

In this tutorial, I'll share my notes and code I've used to setup geospatial Apache Solr searching in Drupal 7 using the Search API module. For this tutorial I created a minimal Ubuntu server virtual machine. All the commands should be executed as a user with permission to modify files, or prefixed with "sudo".

The first thing I do with a fresh virtual machine is check for package upgrades.

$ apt-get update
$ apt-get upgrade

I find it cumbersome to type in a virtual machine window, so I'll install open-ssh and ssh from my Mac. If you plan to do so, you'll need to find your virtual machine's IP address using ifconfig. For this tutorial I added local DNS (/etc/hosts) to point "drupal7.vm" to my VM's IP.

$ apt-get install openssh-server

Install the LAMP stack. The following packages will install Apache httpd as a dependency.

$ apt-get install php5 php5-cli php5-common php5-curl php5-gd php5-mysql php-pear mysql-server

At this point, browsing to your VM/server's IP address will give you the standard Apache welcome message:
It works!
This is the default web page for this server.
The web server software is running but no content has been added, yet.

Install version control.

$ apt-get install git-core

Create a mysql database for Drupal 7.

$ mysql -u youruser -p
mysql> create database drupal7;
mysql> grant all privileges on drupal7.* to 'drupal7'@'localhost' identified by 'somepassword';
mysql> exit

Install drush via Pear.

$ pear upgrade-all
$ pear channel-discover pear.drush.org
$ pear install drush/drush

Verifying drush is installed.

$ which drush
/usr/bin/drush
$ drush --version
drush version 4.5

Create an Apache vhost directory

$ mkdir -p /var/www/vhosts

Download drupal via drush

$ cd /var/www/vhosts
$ drush dl drupal
# rename folder (as necessary)
$ mv drupal-7.10 drupal7

Integrate drupal file system with git

$ cd drupal7
$ git init
$ git add .
$ git commit -am "initial commit of drupal7"

Install drupal via drush

$ drush site-install standard --db-url=mysql://dbuser:pass@localhost/dbname

Add Apache2 vhost

$ cd /etc/apache2/sites-available
# create new file, called "drupal7" with contents:
<VirtualHost *:80>
  ServerName drupal7.vm
  DocumentRoot /var/www/vhosts/drupal7
  ErrorLog /var/log/apache2/drupal7-error_log
  CustomLog /var/log/apache2/drupal7-access_log combined
  <Directory /var/www/vhosts/drupal7>
    AllowOverride All
  </Directory>
</VirtualHost>

# create symlink
$ cd ../sites-enabled
$ ln -s ../sites-available/drupal7 001-drupal7.conf

# enable apache2 mod_rewrite module
$ a2enmod rewrite

# restart apache2
$ /etc/init.d/apache2 restart

At this point, browsing to your VM/server's hostname should show a Drupal installation.

Part 2, Tomcat/Solr

Installing java jdk and tomcat6

$ apt-get install openjdk-6-jdk tomcat6 tomcat6-admin tomcat6-common tomcat6-user

Browsing to your VM/server's hostname on port 8080 (ex: http://drupal7.vm:8080) will show the generic Tomcat welcome message:
It works !
If you're seeing this page via a web browser, it means you've setup Tomcat successfully. Congratulations!

Installing Solr in Tomcat

$ mkdir ~/downloads
$ cd ~/downloads
# Download the latest stable version of Apache Solr from:
url: http://www.apache.org/dyn/closer.cgi/lucene/solr/
# example:
$ wget http://www.motorlogy.com/apache//lucene/solr/3.5.0/apache-solr-3.5.0.tgz
$ tar -xzf apache-solr-3.5.0.tgz

Copy/rename java war file into Tomcat webapps directory

$ cp ~/downloads/apache-solr-3.5.0/dist/apache-solr-3.5.0.war /var/lib/tomcat6/webapps/solr.war

Note: copying the java war file into the Tomcat webapps folder will create this directory automatically:

/var/lib/tomcat6/webapps/solr

Copy solr files

$ cp -r ~/downloads/apache-solr-3.5.0/example/solr/ /var/lib/tomcat6/solr/

Create Catalina config file to link war file to solr directory

$ cd /etc/tomcat6/Catalina/localhost
# create new file: "solr.xml", with the contents:
<?xml version="1.0" encoding="UTF-8"?>
<Context docBase="/var/lib/tomcat6/webapps/solr.war" debug="0" privileged="true" allowLinking="true" crossContext="true">
<Environment name="solr/home" type="java.lang.String" value="/var/lib/tomcat6/solr" override="true" />
</Context>

Setup Tomcat admin user(s)

# edit file: /etc/tomcat6/tomcat-users.xml, ensure similar contents exist:
<?xml version='1.0' encoding='utf-8'?>
<tomcat-users>
<role rolename="admin"/>
<role rolename="manager"/>
<user username="eric" password="supersecretpassword" roles="admin,manager"/>
</tomcat-users>

Update webapps WEB-INF/web.xml file

# edit file: /var/lib/tomcat6/webapps/solr/WEB-INF/web.xml, update "solr/home" section to reflect solr path:
<env-entry>
  <env-entry-name>solr/home</env-entry-name>
  <env-entry-value>/var/lib/tomcat6/solr</env-entry-value>
  <env-entry-type>java.lang.String</env-entry-type>
</env-entry>

Download search api drupal modules that contain solr xml configuration files, and copy into solr conf directory

$ mkdir -p /var/www/vhosts/drupal7/sites/all/modules/contrib
$ cd /var/www/vhosts/drupal7/sites/all/modules/contrib
$ drush dl search_api search_api_solr
$ cp /var/www/vhosts/drupal7/sites/all/modules/contrib/search_api_solr/solrconfig.xml /var/lib/tomcat6/solr/conf/
$ cp /var/www/vhosts/drupal7/sites/all/modules/contrib/search_api_solr/schema.xml /var/lib/tomcat6/solr/conf/

Reset tomcat permissions, and restart tomcat

$ cd /var/lib
$ chown -R tomcat6.tomcat6 tomcat6
$ /etc/init.d/tomcat6 restart

You should now be able to browse to the solr admin java page.
Example: http://drupal7.vm:8080/solr/admin/


If things aren't working well at this point, check the Tomcat logs and look for SEVERE log entries

/var/log/tomcat6/catalina.out

In addition, the solr java module should be listed in the Tomcat Web Application Manager
Ex URL: http://drupal7.vm:8080/manager/html

Part 3, Drupal code

Getting the solr-php-client library from code.google.com

$ mkdir -p /var/www/vhosts/drupal7/sites/all/libraries
$ cd /var/www/vhosts/drupal7/sites/all/libraries

# URL: http://code.google.com/p/solr-php-client/downloads/list
# File: SolrPhpClient.r60.2011-05-04.tgz
$ wget http://solr-php-client.googlecode.com/files/SolrPhpClient.r60.2011-05-04.tgz
$ tar -xzf SolrPhpClient.r60.2011-05-04.tgz

Downloading and installing contrib drupal modules

$ cd /var/www/vhosts/drupal7
$ drush dl entity views ctools facetapi
$ drush en search_api search_api_views search_api_solr search_api_facetapi entity views views_ui ctools facetapi

(Optionally) I install devel, admin_menu, and disable overlay/toolbar

$ drush dl devel admin_menu
$ drush en devel admin_menu
$ drush dis overlay toolbar

Add the tomcat/solr server to Search API configuration.
- URL: /admin/config/search/search_api
- click on "+ Add Server"
- server name: Solr 3.5.0
- Service class: Solr service
- Solr host: localhost
- Solr port: 8080
- Solr path: /solr
- click Create Server

You should receive some confirmation messages:
The server was successfully created.
The Solr server could be reached (latency: # ms).
If not, ensure tomcat/solr is reachable at the url you specified and the tomcat service is running.

At this point Solr is ready to send/receive data and index content, but there is nothing to index. For this tutorial, I decided to build off of user profiles and store latitude and longitude using the geolocation field module.

$ drush dl geolocation
$ drush en geolocation

Adding some user profile fields:
- URL: /admin/config/people/accounts/fields
- First Name | field_name_first | Text
- Last Name | field_name_last | Text
- Geolocation | field_geolocation | Geolocation | Latitude/Longitude

I then added a bunch of users with latitude/longitude coordinates.
- URL: /admin/people/create
- note: I used Google Geocoding API to fetch the coordinates: http://code.google.com/apis/maps/documentation/geocoding/

Adding the search api index.
- URL: /admin/config/search/search_api
- click "+ Add index"
- Index name: People
- Item type: User
- Server: Solr 3.5.0
- click: Create Index

On the next admin page, you can select which fields to index. For this tutorial, I chose: User ID, Name, Email, URL, First Name, and Last Name. Unfortunately, at the time of writing this, the geolocation lat/lng fields are not exposed to the Entity API. I assume this is a temporary problem, and there are numerous patches in the geolocation issue queue.
@see (for example):
Property Info callback for Entity API - http://drupal.org/node/1366642
Fix for Search API not picking up the entity to index it's fields - http://drupal.org/node/1320564

I copied code directly from the issues queue, made some modifications, and created a custom module to expose the geolocation field data to the entity api module. In addition, I added a new property "lat_lon" that concatenates lat and lng together with a comma. @see: http://wiki.apache.org/solr/SpatialSearch

<?php
/**
 * Implements hook_field_info_alter()
 */
function MYMODULE_field_info_alter(&$info) {
  if (isset($info['geolocation_latlng'])) {
    $info['geolocation_latlng']['property_type'] = 'geolocation';
    $info['geolocation_latlng']['property_callbacks'] = array('geolocation_property_info_callback');
  }
}

function geolocation_property_info_callback(&$info, $entity_type, $field, $instance, $field_type) {
  $name = $field['field_name'];
  $property = &$info[$entity_type]['bundles'][$instance['bundle']]['properties'][$name];

  $property['type'] = ($field['cardinality'] != 1) ? 'list<geolocation>' : 'geolocation';
  $property['getter callback'] = 'entity_metadata_field_verbatim_get';
  $property['setter callback'] = 'entity_metadata_field_verbatim_set';
  $property['auto creation'] = 'geolocation_default_values';
  $property['property info'] = geolocation_data_property_info();

  unset($property['query callback']);
}

function geolocation_default_values() {

  return array(
    'lat' => '',
    'lng' => '',
    'lat_sin' => '',
    'last_name' => '',
    'lat_cos' => '',
    'lat_rad' => '',
    'lat_lon' => '',
  );

}

function geolocation_data_property_info($name = NULL) {

  // Build an array of basic property information for the geolocation field.
  $properties = array(
    'lat' => array(
      'label' => t('Latitude'),
    ),
    'lng' => array(
      'label' => t('Longitude'),
    ),
    'lat_sin' => array(
      'label' => t('Sine of Latitude'),
    ),
    'lat_cos' => array(
      'label' => t('Cosine of Latitude'),
    ),
    'lat_rad' => array(
      'label' => t('Radian Latitude'),
    ),
    'lat_lon' => array(
      'label' => t('Latitude,Longitude'),
    ),
  );

  // Add the default values for each of the address field properties.
  foreach ($properties as $key => &$value) {
    
    switch ($key) {
    
      case 'lat_lon':
        $value += array(
          'description' => !empty($name) ? t('!label of field %name', array('!label' => $value['label'], '%name' => $name)) : '',
          'type' => 'text',
          'getter callback' => '_MYMODULE_geolocation_entity_property_verbatim_get',
          'setter callback' => '_MYMODULE_geolocation_entity_property_verbatim_set',
        );
        break;
    
      default:
        $value += array(
          'description' => !empty($name) ? t('!label of field %name', array('!label' => $value['label'], '%name' => $name)) : '',
          'type' => 'text',
          'getter callback' => 'entity_property_verbatim_get',
          'setter callback' => 'entity_property_verbatim_set',
        );
        break;
    
    }

 }

 return $properties;
}


function _MYMODULE_geolocation_entity_property_verbatim_get($data, array $options, $name, $type, $info) {
  if (is_array($data) && isset($data['lat']) && isset($data['lng'])) {
    return $data['lat'] . ',' . $data['lng'];
  }
  return '';
}

function _MYMODULE_geolocation_entity_property_verbatim_set(&$data, $name, $value, $langcode, $type, $info) {
  // TODO
  return;
}
?>

I added this code to a custom module, renamed function calls (as necessary), and enabled. Update the solr index to add the new fields to the index.
- URL: /admin/config/search/search_api/index/people/fields
- Expand "Add Related Fields"
- Choose Geolocation, click Add fields
The above will expose the following fields now available to the index:
- Geolocation » Latitude
- Geolocation » Longitude
- Geolocation » Sine of Latitude
- Geolocation » Cosine of Latitude
- Geolocation » Radian Latitude
- Geolocation » Latitude,Longitude
Enable "Geolocation » Latitude,Longitude" and save changes.

Index the content.
- URL: /admin/config/search/search_api/index/people/status
- Click: Index now
- note: if you had already indexed the content, you'll probably need to clear it first
In my environment, I got the following confirmation message:
Successfully indexed 7 items.

I find it to be very helpful to verify the xml response from Solr directly after making changes to the index/schema.
The following URL structure will query solr for all results and return all fields:

Ex URL: http://drupal7.vm:8080/solr/select/?q=&fl=*

A sample XML document response.

<doc>
  <str name="f_ss_search_api_language"/>
  <str name="f_ss_url">http://drupal7.vm/user/3</str>
  <str name="id">people-3</str>
  <str name="index_id">people</str>
  <long name="is_uid">3</long>
  <str name="item_id">3</str>
  <arr name="spell">
    <str>nashua</str>
    <str>nashua@example.com</str>
    <str>nashua</str>
    <str>nashua</str>
    <str>42.933692,-72.278141</str>
  </arr>
  <str name="ss_search_api_id">3</str>
  <str name="ss_search_api_language"/>
  <str name="ss_url">http://drupal7.vm/user/3</str>
  <arr name="t_field_geolocation:lat_lon">
    <str>42.933692,-72.278141</str>
  </arr>
  <arr name="t_field_name_first">
    <str>nashua</str>
  </arr>
  <arr name="t_field_name_last">
    <str>nashua</str>
  </arr>
  <arr name="t_mail">
    <str>nashua@example.com</str>
  </arr>
  <arr name="t_name">
    <str>nashua</str>
  </arr>
</doc>

Take note the field name in the following XML, it is used in the next file edit.

<arr name="t_field_geolocation:lat_lon">
  <str>42.933692,-72.278141</str>
</arr>

Update the solr schema.xml configuration and add the geospatial fieldType and field data.

# Edit file: /var/lib/tomcat6/solr/conf/schema.xml
# Just prior to the closing "</types>" tag, I inserted: (around line 287)
    <fieldType name="point" class="solr.PointType" dimension="2" subFieldSuffix="_d"/>
    <fieldType name="location" class="solr.LatLonType" subFieldSuffix="_coordinate"/>
    <fieldtype name="geohash" class="solr.GeoHashField"/>

# And, just after the opening "<fields>" tag, I inserted: 
    <field name="t_field_geolocation:lat_lon" type="location" indexed="true" stored="true"/>
    <dynamicField name="*_coordinate"  type="tdouble" indexed="true"  stored="false"/>

Restart Tomcat

$ /etc/init.d/tomcat6 restart

Since the schema and solr data types have been updated, the content will have to be re-indexed.
- URL: /admin/config/search/search_api/index/people/status
- click: Clear index
- click: Index now

Returning to the solr query above will now show updated xml: (note: no longer an array)

<str name="t_field_geolocation:lat_lon">42.933692,-72.278141</str>

Verify the native solr geospatial searching is working using the following query syntax:
URL: http://drupal7.vm:8080/solr/select/?q=&fl=*&fq={!geofilt sfield=t_field_geolocation:lat_lon pt=42.933692,-72.278141 d=100}
By putting a distance parameter of 100 (kilometers) and Nashua NH coordinates, I get 2 results: Nashua and Portsmouth, awesome.

Create a solr integrated view.
- URL: /admin/structure/views/add
- View name: People
- Show: People
- Create a Page [checked]
- Path: people
- Continue & edit
Note: at this point, you have full reign over view configuration. For this tutorial, I set the format to Grid, and added some fields:
- Geolocation: Latitude,Longitude (indexed)
- Indexed User: Email
- Indexed User: First Name
- Indexed User: Last Name
- Indexed User: Name
Save the view when edits are complete.

Browsing to the view will show something like this:
Ex URL: http://drupal7.vm.people


The next chunk of custom code modifies the solr query executed and adds geospatial filtering.
@see: hook_search_api_solr_query_alter(array &$call_args, SearchApiQueryInterface $query)

<?php
function MYMODULE_search_api_solr_query_alter(array &$call_args, SearchApiQueryInterface $query) {

  $lat = 42.933692;
  $lng = -72.278141;
  $distance = 100;

  $call_args['params']['fq'][] = "{!geofilt sfield=t_field_geolocation:lat_lon pt={$lat},{$lng} d={$distance}}";

}
?>

The above code will limit the view's results using the hardcoded coordinates.


Clearly, it works but there are loose ends to tie..
- automatically fetch a user's coordinates to store in the geolocation field
- add a search form to the people view page to allow the user to search for a location (instead of hard coded coordinates, blah)
- translate the user's location search input to coordinates using an API

Hopefully, I can find more time to elaborate on this tutorial in the near future! Cheers.

In this article, I'll show an example of how to implement a Subversion pre-commit hook to integrate with Drupal. Pre-commit hooks can be used to execute any arbitrary code, such as deployment procedures, archiving databases, etc. For this example, I will show how to check for the creation of a subversion tag and archive the database.

To get started, I created a local subversion repository.

# create folder for subversion repositories
$ mkdir /var/subversion

# create the subversion repository
$ svnadmin create /var/subversion/project

Upon creating a new local svn repository, a hooks directory will be created. Example: /var/subversion/project/hooks

Inside this directory will be a bunch of sample scripts ending in ".tmpl" which contain example hook scripts. Here are the contents of the example pre-commit hook without comments:

$ cat pre-commit.tmpl | egrep -iv "(^#|^$)"
REPOS="$1"
TXN="$2"
SVNLOOK=/usr/bin/svnlook
$SVNLOOK log -t "$TXN" "$REPOS" | \
   grep "[a-zA-Z0-9]" > /dev/null || exit 1
commit-access-control.pl "$REPOS" "$TXN" commit-access-control.cfg || exit 1
exit 0

As noted in the pre-commit.tmpl file, there are 2 arguments being passed to the pre-commit script:

[1] REPOS-PATH   (the path to this repository)
[2] TXN-NAME     (the name of the txn about to be committed)

I created a new file called "pre-commit" and added the following contents:

#!/bin/bash

/var/www/vhosts/project.vm/scripts/svn-pre-commit.php "$1" "$2"

I then made the file executable.

$ chmod ug+w pre-commit

The above script simply passes the arguments to a PHP script contained with the Drupal project.

In my scripts folder (/var/www/vhosts/project.vm/scripts), I created the PHP script "svn-pre-commit.php" with the following contents:

#!/usr/bin/php
<?php

// get args
$repo = $argv[1];

// define path to mysql backups
$mysql_backups_path = '/var/www/vhosts/project.vm/database';

// define path to drupal docroot
$drupal_docroot_path = '/var/www/vhosts/project.vm/htdocs';

// get changed path
// example output:
// A   tags/20110503/
$svn_look = `svnlook changed $repo`;

// define pattern to break apart svnlook changed
$pattern = '/^\s*([A-Za-z])\s*(.*)$/';

// execute preg match
preg_match($pattern, $svn_look, $matches);
$svn_action = $matches[1];
$svn_changed_path = $matches[2];

// check if a tag is being created
if ($svn_action == 'A' && substr($svn_changed_path, 0, 5)=='tags/') {

  // get tag name
  $exploded = explode('/', $svn_changed_path);
  $tag_name = $exploded[1];
  
  // change dir to drupal docroot
  chdir($drupal_docroot_path);

  // backup mysql database using drush
  `/var/www/drush/drush sql-dump > {$mysql_backups_path}/tag_{$tag_name}.sql`;

}

I also made this script executable:

$ chmod ug+w svn-pre-commit.php

Now, assuming that my Drupal site is integrated with the subversion repository, and development is at a point to deploy/create a new tag, I executed the following command to create the subversion tag:

$ svn cp file:///var/subversion/project/trunk file:///var/subversion/project/tags/beta-0.1 -m "creating beta 0.1 tag"

To verify, I entered the directory containing my database dumps to checkout the result:

$ cd /var/www/vhosts/project.vm/database

$ ls -1
tag_beta-0.1.sql

In this article, I'll show the commands I have been using to set up a fresh Centos server, configured for Apache, MySQL, PHP, Tomcat, Drupal, and Apache Solr. For my article, I used Parallels to create a virtual machine from the Centos 5.6 64bit ISOs I downloaded. To simply this article, all commands are being executed as root, firewall configurations and performance tweaks are not accounted for.

Once the distribution is installed, the first thing I do is upgrade all packages.

$ yum update

Install PHP, Apache, and MySQL

$ yum install php53 php53-gd php53-mbstring php53-mysql php53-xml mysql-server httpd

Set runlevels for Apache and MySQL

$ chkconfig --level 2345 httpd on
$ chkconfig --level 2345 mysqld on

Install subversion. I chose to use subversion for this article because the Drupal 6.x installation works well with svn:externals to fetch the SolrPhpClient library. All subversion commands are connecting to a local subversion repository. If you are using an external server (like Beanstalk), you will have to transpose all commands from using "file://" to "https://".

$ yum install subversion

Add a new local subversion repository (OPTIONAL).

$ mkdir /var/subversion
$ svnadmin create /var/subversion/example.com
$ svn mkdir file:///var/subversion/example.com/trunk -m "added trunk"
$ svn mkdir file:///var/subversion/example.com/branches -m "added branches"
$ svn mkdir file:///var/subversion/example.com/tags -m "added tags"

Download/setup drush

$ cd /var/www
$ wget http://ftp.drupal.org/files/projects/drush-7.x-4.4.tar.gz
$ tar -xzf drush-7.x-4.4.tar.gz
$ ln -s /var/www/drush/drush /usr/local/bin/drush

Create a vhost location on the server for the Drupal installation.

$ mkdir /var/www/vhosts
$ cd /var/www/vhosts
$ drush dl drupal
$ mv drupal-7.0/ example.com 

Integrate the Drupal files with subversion

$ cd /var/www/vhosts/example.com
$ svn co file:///var/subversion/example.com/trunk .
$ svn add * .htaccess
$ svn commit -m "downloaded drupal"

Download the Drupal apachesolr module

# make a folder for contrib modules
$ mkdir /var/www/vhosts/example.com/sites/all/modules/contrib
$ cd /var/www/vhosts/example.com/sites/all/modules/contrib

# note: in the below command, you may be prompted to choose which version of the Solr module to install. I choose option 2 for the Supported version
$ drush dl apachesolr

# commit to subversion
$ cd /var/www/vhosts/example.com/sites/all/modules
$ svn add contrib
$ svn commit -m "added contrib folder and apachesolr module"

Setup MySQL

# start mysql
$ /etc/init.d/mysqld start

# set root password
$ /usr/bin/mysqladmin -u root password 'new-password'

# create new database, user, and set permissions
$ mysql --execute="create database db_example"
$ mysql --execute="grant all privileges on db_example.* to 'example-user'@'localhost' identified by 'some_password'"

Setup Apache vhost

$ cd /etc/httpd/conf.d

# create a new file "example.com.conf", with the contents:

NameVirtualHost *:80

<Directory /var/www/vhosts>
  AllowOverride All
</Directory>

<VirtualHost *:80>
  ServerName example.com
  DocumentRoot /var/www/vhosts/example.com
  ErrorLog logs/example.com-error_log
  CustomLog logs/example.com-access_log common
</VirtualHost>

Reset Apache file permissions. NOTE: you will need a more solid/secure configuration for this!

$ cd /var/www
$ chown -R apache.apache drush*
$ chown -R apache.apache vhosts

Start Apache

$ /etc/init.d/httpd start

Install Drupal via drush

$ cd /var/www/vhosts/example.com

# note: you can set your user 1 username, password, email, etc in the following command if desired. type "drush help si" for more install options
$ drush site-install standard --sites-subdir=example.com --db-url=mysqli://example-user:some_password@localhost/db_example

At this point, you should be able to browse to your site and it will be up and running.


Now, we move onto Tomcat and Solr!

Installing Tomcat and Java. The default Centos yum repositories provide Tomcat5. I prefer Tomcat6, so there are some extras steps below and a dependency issue I had to resolve.

# added repo file to get tomcat6:
$ cd /etc/yum.repos.d/
$ wget http://www.jpackage.org/jpackage50.repo

# install Java JDK:
$ yum install java-1.6.0-openjdk

# install tomcat6:
$ yum install tomcat6 tomcat6-admin-webapps tomcat6-webapps

# dang, dependency issue... (!)

java-1.4.2-gcj-compat-1.4.2.0-40jpp.115.x86_64 from base has depsolving problems
  --> Missing Dependency: /usr/bin/rebuild-security-providers is needed by package java-1.4.2-gcj-compat-1.4.2.0-40jpp.115.x86_64 (base)
Error: Missing Dependency: /usr/bin/rebuild-security-providers is needed by package java-1.4.2-gcj-compat-1.4.2.0-40jpp.115.x86_64 (base)
 You could try using --skip-broken to work around the problem
 You could try running: package-cleanup --problems
                        package-cleanup --dupes
                        rpm -Va --nofiles --nodigest
The program package-cleanup is found in the yum-utils package.

# Fixing dependency issue (OPTIONAL):
$ mkdir ~/downloads
$ cd ~/downloads
$ wget http://plone.lucidsolutions.co.nz/linux/centos/images/jpackage-utils-compat-el5-0.0.1-1.noarch.rpm
$ rpm -Uvh jpackage-utils-compat-el5-0.0.1-1.noarch.rpm
$ yum install tomcat6 tomcat6-admin-webapps tomcat6-webapps

# setting tomcat runlevels
$ chkconfig --level 2345 tomcat6 on

# starting tomcat
$ /etc/init.d/tomcat6 start

At this point, you should be able to access Tomcat in your browser (http://example.com:8080)


Downloading Solr Java library.

$ cd ~/downloads
# note: you may need to choose a different mirror to download
$ wget http://www.fightrice.com/mirrors/apache//lucene/solr/1.4.1/apache-solr-1.4.1.tgz
$ tar -xzf apache-solr-1.4.1.tgz

# copy/rename solr war file into Tomcat webapps directory
$ cp ~/downloads/apache-solr-1.4.1/dist/apache-solr-1.4.1.war /var/lib/tomcat6/webapps/solr.war

# copy solr files
$ cp -r ~/downloads/apache-solr-1.4.1/example/solr/ /var/lib/tomcat6/solr/

Copying the java war file into the Tomcat webapps folder will create this directory automatically:

/var/lib/tomcat6/webapps/solr

Create Catalina config file to link war file to solr directory:

$ cd /etc/tomcat6/Catalina/localhost

# create new file: "solr.xml", with the contents:

<?xml version="1.0" encoding="UTF-8"?>
<Context docBase="/var/lib/tomcat6/webapps/solr.war" debug="0" privileged="true" allowLinking="true" crossContext="true">
<Environment name="solr/home" type="java.lang.String" value="/var/lib/tomcat6/solr" override="true" />
</Context>

Setup Tomcat admin user(s):

# edit file: /etc/tomcat6/tomcat-users.xml, ensure similar contents exist:

<?xml version='1.0' encoding='utf-8'?>
<tomcat-users>
<role rolename="admin"/>
<role rolename="manager"/>
<user username="eric" password="supersecretpassword" roles="admin,manager"/>
</tomcat-users>

Update WEB-INF/web.xml file:

# edit file: /var/lib/tomcat6/webapps/solr/WEB-INF/web.xml, update section to reflect solr path:

<env-entry>
  <env-entry-name>solr/home</env-entry-name>
  <env-entry-value>/var/lib/tomcat6/solr</env-entry-value>
  <env-entry-type>java.lang.String</env-entry-type>
</env-entry>

Copy conf files from Drupal apachesolr module into Tomcat Solr conf directory (overwrite):

$ cp /var/www/vhosts/example.com/sites/all/modules/contrib/apachesolr/protwords.txt /var/lib/tomcat6/solr/conf/
$ cp /var/www/vhosts/example.com/sites/all/modules/contrib/apachesolr/schema.xml /var/lib/tomcat6/solr/conf/
$ cp /var/www/vhosts/example.com/sites/all/modules/contrib/apachesolr/solrconfig.xml /var/lib/tomcat6/solr/conf/

Reset Tomcat permissions/ownership:

$ cd /var/lib
$ chown -R tomcat.tomcat tomcat6/

Restart Tomcat

$ /etc/init.d/tomcat6 restart

At this point, you should be able to access the solr/admin tomcat Page (http://example.com:8080/solr/admin)


If things are not working well at this point, check the Tomcat logs:

/var/log/tomcat6/catalina.out

And, ensure the solr java module is listed in the Tomcat Web Application Manager: http://example.com:8080/manager/html

If all is well, you can now enable the Drupal apachesolr modules:

$ cd /var/www/vhosts/example.com
$ drush en apachesolr apachesolr_search apachesolr_taxonomy apachesolr_access --uri=example.com

Log into your Drupal site. NOTE: default account (via drush): admin/admin

Edit default Apache Solr Host Settings:
URL: http://example.com/admin/config/search/apachesolr/server/solr/edit
Change url to: http://example.com:8080/solr, and save form.

Go to Drupal search settings page:
URL: http://example.com/admin/config/search/settings
Change the default search mode to "Apache Solr search", and save form.

Now, you are ready to test the indexing and integration.

Add a new piece of content to test indexing.
# Example:
# http://example.com/node/add/article
# title: Test Article
# Body: test test test

Browse to solr index page:
URL: http://example.com/admin/config/search/apachesolr/index
Select: Index queued content, and click on Begin button
You should see a status message like: "1 item processed successfully." and "Number of documents in index: 0 (1 sent but not yet processed)"
A few minutes later, refreshing the index page should show: "Number of documents in index: 1"

Search for "test" to verify Solr results.
URL: http://example.com/search/site/test


You can also review search results via solr/admin
URL: http://example.com:8080/solr/admin/
Enter "test" in query string box and click search

Part 2, Multicore Configuration (OPTIONAL)

If you need to run multiple sites off a single Solr Tomcat installation, you can setup multicore..

Copy the multicore xml file into your solr directory:

$ cp ~/downloads/apache-solr-1.4.1/example/multicore/solr.xml /var/lib/tomcat6/solr/

Create a new directory for each multisite in the solr directory:

$ mkdir /var/lib/tomcat6/solr/example.com

Replicate the solr conf directory into the new multisite directory:

cp -r /var/lib/tomcat6/solr/conf /var/lib/tomcat6/solr/example.com/conf/

Update the solr.xml file:

# edit file: /var/lib/tomcat6/solr/solr.xml, added <core> section for each site:
<cores adminPath="/admin/cores">
  <core name="example.com" instanceDir="example.com" />
</cores>

Restart Tomcat

$ /etc/init.d/tomcat6 restart

Now, your new multicore site will be accessible here: http://example.com:8080/solr/example.com/admin/