Avatar-eric-london
Created by Eric London on 2013-04-18
Tags:
 
Please note: the content on this page orginates from ericlondon.com.
I am running Apache + Passenger on a server which is set to use the global RVM gemset for all vhosts. Here's a simple cron script to check for gem updates.

Check which RVM gemset you are using:

$ rvm current
ruby-1.9.3-p392@global

# search through apache conf to find RVM environment path:
$ grep -i passenger /etc/apache2/apache2.conf
LoadModule passenger_module /usr/local/rvm/gems/ruby-1.9.3-p392@global/gems/passenger-3.0.19/ext/apache2/mod_passenger.so
PassengerRoot /usr/local/rvm/gems/ruby-1.9.3-p392@global/gems/passenger-3.0.19
PassengerRuby /usr/local/rvm/wrappers/ruby-1.9.3-p392@global/ruby


Create a new script: (ex: /root/scripts/cron_gem_outdated.sh)

#!/usr/bin/env bash
source /usr/local/rvm/environments/ruby-1.9.3-p392@global
gem outdated


Add cron job:

$ sudo crontab -e
1 1 * * * /root/scripts/cron_gem_outdated.sh
Please note: the content on this page orginates from ericlondon.com.
Recently I've been tracking price drops in iTunes apps, so I thought I'd roll my own feed parser in Ruby and share the code. In this blog post I'll demonstrate the following: MongoDB for document database storage, MongoID for the Ruby library, Curb for feed fetching, Nokogiri/Nori for XML parsing, Sinatra for a simple web server, and Google charts for a price spark image.

Installed MongoDB via Homebrew

# install 
brew install mongodb

# started service
mongod


Created a new project Gemfile, file: Gemfile

source 'http://rubygems.org'

gem 'mongoid', '~> 3.0'
gem 'curb'
gem 'nokogiri'
gem 'nori'
gem 'sinatra'
gem 'googlecharts'


Installed gems

bundle


Created a MongoID config file, file: mongoid.yml

development:
  sessions:
    default:
      database: itunes_feeds
      hosts:
        - localhost:27017


Created a mongo include file to define class structure for mongo objects, file: mongo.rb

require 'mongoid'

# load mongo conf
Mongoid.load!('mongoid.yml', :development)

class FeedItem
  include Mongoid::Document
  embeds_many :feedItemPrices
end

class FeedItemPrice
  include Mongoid::Document
  embedded_in :feedItem
end


Created a simple class to fetch, parse, and store the iTunes feed data, file: itunes_feed_fetcher.rb

require 'rubygems'
require 'curb'
require 'nokogiri'
require 'nori'

require './mongo.rb'

class ItunesFeedFetcher

  def initialize
    @feed_count = 300
    @feed_url = "https://itunes.apple.com/us/rss/toppaidapplications/limit=#{@feed_count}/xml"
    @feed_items_new = 0
    @feed_items_updated = 0
  end

  def fetch

    # curl feed url
    curld = Curl::Easy.perform @feed_url

    # convert rss feed xml to hash
    nori = Nori.new(:parser => :nokogiri)
    @feed_data = nori.parse curld.body_str

  end

  def process

    return nil if @feed_data.nil?

    @feed_data['feed']['entry'].each do |entry|

      # get itunes id
      itunes_id = entry['id']

      # check if entry exists in database
      existing = FeedItem.where(itunes_id: itunes_id).first

      if !existing.nil?

        # check if entry has been updated
        if entry['updated'].utc >= existing['updated'].utc
          # todo: update entry details
        end

        # get entry price, minus dollar sign
        entry_price = entry['im:price'].scan(/[0-9.]+/).first

        # create feed item price record
        fip = existing.feedItemPrices.create({created: Time.now, price: entry_price})
        fip.save

        @feed_items_updated += 1

      else

        # get entry price, minus dollar sign
        entry_price = entry['im:price'].scan(/[0-9.]+/).first

        # remove entry price, will be embedded instead
        entry.delete 'im:price'

        # set itunes id to entry
        entry['itunes_id'] = itunes_id

        # create new feed item record
        fi = FeedItem.new entry
        fi.save

        # create feed item price record
        fip = fi.feedItemPrices.create({created: Time.now, price: entry_price})
        fip.save

        @feed_items_new += 1

      end

    end

  end

  def report
    "New: #{@feed_items_new}<br/>Updated: #{@feed_items_updated}"
  end

end


Created a simple sinatra website with 2 urls ("/" and "/fetch"), file: sinatra.rb

#!/usr/bin/env ruby

require 'rubygems'
require 'sinatra'
require 'gchart'

require './mongo.rb'
require './itunes_feed_fetcher.rb'

get '/' do

  output = '<table>'
  FeedItem.each do |fi|
    output += "<tr>"
    output += "<td><img src='#{fi['im:image'][0]}' /></td>"
    output += "<td><b>#{fi['im:name']}</b></td>"
    #output += "<td>#{fi['content']}</td>"

    # collect feed item prices
    prices = fi.feedItemPrices.collect {|fip| fip['price'].to_f}

    # create google chart image url
    chart_url = Gchart.sparkline(:data => prices, :size => '120x40', :line_colors => '0077CC')
    output += "<td><img src='#{chart_url}' /></td>"

    output += "</tr>"
  end
  output += "</table>"
  output

end

get '/fetch' do
  iff = ItunesFeedFetcher.new
  iff.fetch
  iff.process
  iff.report
end


I then started the sinatra app:

./sinatra.rb


Browsing to http://localhost:4567/fetch fetches, parses, and stores the data in mongo. sample output:
New: 0
Updated: 300

And, browsing to http://localhost:4567/ shows the feed items with a price spark image. As you can see I randomized the price spark data to make it more interesting.
itunes feeds

Source code on Github
Please note: the content on this page orginates from ericlondon.com.
In this blog article, I'll demonstrate a proof of concept: MySQL replication to a Redis cache server. To outline the components used:
  • Ubuntu - linux
  • Mysql - primary database
  • Redis - cache database
  • Ruby - Gearman worker + client
  • Gearman - job queue service
  • MySQL JSON UDF
  • MySQL Gearman UDF
At FreePriceAlerts.com we implemented a similar configuration to produce high volume streams of data for Ziftr.

I started with a clean and minimal installation of Ubuntu 12.10 Server.

# upgrade packages
apt-get update
apt-get upgrade -y

# install ssh server
apt-get install openssh-server -y


Installed Mysql Server

apt-get install mysql-server -y


Install Ruby via RVM

curl -L https://get.rvm.io | bash -s stable
source /etc/profile.d/rvm.sh
# rvm requirements...
apt-get --no-install-recommends install build-essential openssl libreadline6 libreadline6-dev curl git-core zlib1g zlib1g-dev libssl-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt-dev autoconf libc6-dev libgdbm-dev ncurses-dev automake libtool bison subversion pkg-config libffi-dev
rvm install ruby


Install Redis

apt-get install tcl8.5 -y
wget http://redis.googlecode.com/files/redis-2.6.9.tar.gz
tar -xzf redis-2.6.9.tar.gz
cd redis-2.6.9
make
make test
cd src
cp redis-benchmark redis-check-aof redis-check-dump redis-cli redis-sentinel redis-server /usr/local/bin/
cd ..
cp redis.conf /etc

# start redis
redis-server /etc/redis.conf &

# test redis
redis-cli ping
PONG


Install Gearman

apt-get install gearman gearman-server -y

# check if service is running
/etc/init.d/gearman-job-server status
 * gearmand is running


Installed MySQL JSON UDF

apt-get install libmysqlclient-dev -y
cd
mkdir ~/lib_mysqludf_json
cd ~/lib_mysqludf_json
wget http://www.mysqludf.org/lib_mysqludf_json/lib_mysqludf_json_0.0.2.tar.gz
tar -xzf lib_mysqludf_json_0.0.2.tar.gz

# remove shared object, and recompile
rm lib_mysqludf_json.so
gcc $(mysql_config --cflags) -shared -fPIC -o lib_mysqludf_json.so lib_mysqludf_json.c

# locate plugin directory
mysql -u root -pPASSWORD --execute="show variables like '%plugin%';"
+---------------+------------------------+
| Variable_name | Value                  |
+---------------+------------------------+
| plugin_dir    | /usr/lib/mysql/plugin/ |
+---------------+------------------------+

# copy shared object to plugin directory
cp lib_mysqludf_json.so /usr/lib/mysql/plugin/

# enable json_object method
mysql -u root -pPASSWORD --execute="create function json_object returns string soname 'lib_mysqludf_json.so'"


Installed MySQL Gearman UDF

apt-get install libgearman-dev -y
cd
wget https://launchpad.net/gearman-mysql-udf/trunk/0.6/+download/gearman-mysql-udf-0.6.tar.gz
tar -xzf gearman-mysql-udf-0.6.tar.gz
cd gearman-mysql-udf-0.6
./configure --with-mysql=/usr/bin/mysql_config --libdir=/usr/lib/mysql/plugin/
make
make install

# enabled udf functions
mysql -u root -pPASSWORD --execute="CREATE FUNCTION gman_do_background RETURNS STRING SONAME 'libgearman_mysql_udf.so'"
mysql -u root -pPASSWORD --execute="CREATE FUNCTION gman_servers_set RETURNS STRING SONAME 'libgearman_mysql_udf.so'"

# set gearman server
mysql -u root -pPASSWORD --execute="SELECT gman_servers_set('127.0.0.1')"


Ruby setup code

# setup RVM gemset
mkdir ~/ruby
echo "rvm use --create ruby-1.9.3@redis_gearman" > ~/ruby/.rvmrc
cd ~/ruby

# new file: Gemfile; contents:
source 'https://rubygems.org'
gem 'gearman-ruby'
gem 'redis'

# execute bundle to install gems
bundle


Created redis gearman worker, new file: redis_worker.rb

#!/usr/bin/env ruby

require 'rubygems'
require 'gearman'
require 'redis'
require 'json'

servers = ['localhost']
worker = Gearman::Worker.new(servers)

REDIS_DELIMITER = ':'
$redis = Redis.new

module RedisWorker
  def RedisWorker.work(data, job)

    # decode json
    json_data = JSON.parse data

    # create redis key
    redis_key = "user_page_views#{REDIS_DELIMITER}#{json_data['user_id']}"

    $redis.lpush redis_key, data

    true
  end
end

worker.add_ability('redis_worker') do |data,job|
  RedisWorker::work data,job
end

loop {worker.work}


Set file executable

chmod +x redis_worker.rb


Created redis gearman [test] client, new file: redis_client.rb

#!/usr/bin/env ruby

require 'rubygems'
require 'gearman'
require 'json'

servers = ['localhost']
client = Gearman::Client.new(servers)
taskset = Gearman::TaskSet.new(client)

data = '{"user_id":1,"timestamp":"2013-02-14 19:13:15","page":"http://www.google.com"}'

result = client.do_task('redis_worker', data)
puts result


Set file executable

chmod +x redis_client.rb


Testing Gearman worker & Redis

# in terminal 1, start worker
./redis_worker.rb

# in terminal 2, check gearman status & verify worker
(echo status ; sleep 0.1) | netcat 127.0.0.1 4730
redis_worker	0	0	1

# in terminal 3, monitor redis
redis-cli monitor
OK

# in terminal 4, run client test script
./redis_client.rb 
true

# in terminal 3, verify redis lpush:
redis-cli monitor
OK
1361012555.700504 [0 127.0.0.1:34135] "lpush" "user_page_views:1" "{\"user_id\":1,\"timestamp\":\"2013-02-14 19:13:15\",\"page\":\"http://www.google.com\"}"


MySQL data setup

# add database & table
mysql -u root -pPASSWORD
mysql> create database redisgearman;
mysql> use redisgearman;
mysql> CREATE TABLE `user_page_views` (
  `id` int(11) unsigned NOT NULL AUTO_INCREMENT,
  `user_id` int(11) NOT NULL,
  `timestamp` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  `page` varchar(255) NOT NULL DEFAULT '',
  PRIMARY KEY (`id`),
  KEY `user_id` (`user_id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
mysql> exit

# add test record
mysql -u root -pPASSWORD redisgearman --execute="insert into user_page_views (user_id, page) values (1, 'http://www.google.com')"

# ensure json udf is working
mysql -u root -pPASSWORD redisgearman --execute="select json_object(user_id as \`user_id\`, timestamp as \`timestamp\`, page as \`page\`) as json from user_page_views"
+--------------------------------------------------------------------------------+
| json                                                                           |
+--------------------------------------------------------------------------------+
| {"user_id":1,"timestamp":"2013-02-14 19:13:15","page":"http://www.google.com"} |
+--------------------------------------------------------------------------------+


Add MySQL trigger, new file: ~/trigger.sql

DELIMITER $$
CREATE TRIGGER redisgearman.redis_gearman AFTER INSERT ON redisgearman.user_page_views
  FOR EACH ROW BEGIN
    SET @ret=gman_do_background('redis_worker', json_object(NEW.user_id as `user_id`, NEW.timestamp as `timestamp`, NEW.page as `page`)); 
  END$$
DELIMITER ;


Enable trigger

mysql -u root -pPASSWORD redisgearman < ~/trigger.sql


Trigger + Gearman worker + Redis test.

# example insert statement
insert into user_page_views (user_id, page) values (1, 'http://ericlondon.com/recent-posts');

# mysql insert => mysql trigger => gearman udf => ruby redis worker => redis insert..

# output from redis-cli monitor
redis-cli monitor
OK
1361327500.888805 [0 127.0.0.1:33649] "lpush" "user_page_views:1" "{\"user_id\":1,\"timestamp\":\"2013-02-19 21:31:40\",\"page\":\"http://ericlondon.com/recent-posts\"}"


Now the MySQL table replicates to the Redis cache database. For future queries, check Redis first, and fall back on MySQL. Here's an example class to do so:

#!/usr/bin/env ruby

require 'rubygems'
require 'redis'
require 'json'
require 'mysql2'

class RedisMysql

  def initialize
    @redis = Redis.new
    @mysql = Mysql2::Client.new(:host => 'localhost', :username => 'root', :password => 'PASSWORD', :database => 'redisgearman')
    @redis_results = []
  end

  def query(key, limit)
    @redis_results = query_redis key,limit
    return @redis_results if @redis_results.size >= limit

    @mysql_results = query_mysql key, (limit-@redis_results.size)
    @redis_results.concat @mysql_results

  end

  def query_redis(key, limit)
    results = @redis.lrange key, 0, limit
    return [] if results.nil?
    results.collect {|r| JSON.parse r}
  end

  def query_mysql(key, limit)

    # parse args
    parts = key.split ':'
    mysql_table = parts[0]
    user_id = parts[1]

    # get last timestamp from redis results
    last_timestamp = @redis_results.last['timestamp'] unless @redis_results.empty?

    where = []
    where << "user_id = '#{@mysql.escape user_id}'"
    where << "timestamp < '#{@mysql.escape last_timestamp}'" unless last_timestamp.nil?

    sql = "
      select *
      from user_page_views
      where #{where.join ' and '}
      order by id desc
      limit #{limit}"

    results = @mysql.query sql
    return [] if results.nil?
    results.collect {|r| r}

  end
end


Class usage

rm = RedisMysql.new
results = rm.query 'user_page_views:1', 10

# debug
puts results


Source code on GitHub
Please note: the content on this page orginates from ericlondon.com.
In this tutorial, I'll show how I created an iOS app to make a JSON API request to a Rails app and show the content as images in the iPad simulator.

In my rails app model file, I defined an "as_json" method to format the JSON output:
# file: app/models/image.rb

def as_json(options={})
  {
    :thumb_path => self.upload.url(:thumb),
    :image_desc => self.description.blank? ? self.title : "#{self.title}\n#{self.description}",
    :large_path => self.upload.url(:large),
  }
end


And in my images controller, I setup my index method for JSON output:
# file: app/controllers/images_controller.rb

class ImagesController < ApplicationController

  def index
    // ... snip...
    respond_to do |format|
      format.html {  }
      format.json { render json: @images }
    end

  end

end


Onto Xcode. I created a new project:

File > New > Project
- iOS > Single View Application; Next

I entered the following project options:
- Product Name: Pictures
- Organization Name: self
- Company Identifier: com.self
- Class Prefix: Pictures
- Devices: iPad
- Use Storyboards: yes
- User Automatic Reference Counting: yes
- Include Unit Tests: (optional, not used in this tutorial)

Click Next; Choose Location; Click Create

For this project, I decided to use CocoaPods to manage my third party objc libraries. In this case, primarily AFNetworking.

In the terminal, go to dir that contains the new .xcodeproj
example: /Users/Eric/Documents/Objective-C/Pictures

Create new file: .rvmrc (if you're using RVM)

rvm use --create ruby-1.9.3@iOS.Pictures


Create new Gemfile with contents:

source 'https://rubygems.org'
gem 'cocoapods'


Execute bundle to fetch gems

bundle


Create new Podfile with contents:

platform :ios, '6.0'
pod 'FMDB'
pod 'AFNetworking'
xcodeproj 'Pictures.xcodeproj'


Execute pod install to get objc libraries and setup Xcode workspace

pod install


You should now have the following file structure:

$ ls -1
Gemfile
Gemfile.lock
Pictures
Pictures.xcodeproj
Pictures.xcworkspace
Podfile
Podfile.lock
Pods


From now on, open "Pictures.xcworkspace" instead of "Pictures.xcodeproj". You'll see the following structure:
pictures workspace

To revise the default storyboard and add a new collection view controller:
- click on MainStoryboard.storyboard
- click on Pictures View Controller, and press delete
pictures view controller
- drag a Collection View Controller onto the canvas

Edit PicturesViewController.h

// change:
@interface PicturesViewController : UIViewController
// to:
@interface PicturesViewController : UICollectionViewController


Click on MainStoryboard.storyboard. Click on Collection View Controller (in Collection View Controller Scene). In the Utilities region, click on Identity inspector, enter "PicturesViewController" for the custom class.
pictures view controller class

Click on Collection View Cell.
collection view cell
File >> New >> File… >> in iOS >> Cocoa Touch >> Objective-C class >> Next
- Class: PicturesCollectionViewCell
- Subclass of: UICollectionViewCell
- Next >> Create
The above commands will create the files: PicturesCollectionViewCell.h and PicturesCollectionViewCell.m

Click on PicturesViewController.m, add:

#import "PicturesCollectionViewCell.h"


Click on MainStoryboard.storyboard. Click on Collection View Cell in Pictures View Controller Scene. In the Utilities region, click on Identity inspector, enter "PicturesCollectionViewCell" for the custom class
pictures collection view cell

Click on Pictures Collection View Cell in Pictures View Controller Scene. In the Utilities region, click on Attributes inspector, under Collection Reusable View, enter "pictureCell" in the Identifier field.
picture cell

Drag a new Image View object inside the collection view cell.

Click on the button to show the Assistant editor. This will allow you to connect elements from the interface builder to code in your classes. In the Pictures View controller scene, click on Image View. In the code window to the right, you should see the PicturesViewController.h file. You'll want to change that to the PicturesCollectionViewCell.h header file. Clicking on the class name will allow you to change the active file.
imageview select file

Hold down the control key and drag UIImage view object to the PicturesCollectionViewCell.h header file; release inside the @interface and @end lines of code.
imageview drag

In the popup, enter:
- Connection: Outlet
- Name: pictureImageView
- Type: UIImageView
- Storage: Weak
Click Connect.

The above commands will add the following IBOutlet code:

@property (weak, nonatomic) IBOutlet UIImageView *pictureImageView;


Now it's time to add the guts of the view controller code. Click on PicturesViewController.m file.

TIP:
With the Utilities View open, click on the Quick Help tab. If you click on "PicturesViewController" in the @interface PicturesViewController () line of code, the quick help section will provide a reference link to the class that the PicturesViewController inherits from: UICollectionViewController. Click on "UICollectionViewController Class Reference" link. In the "Conforms to" section at the top, click on UICollectionViewDataSource. As noted in the documentation, "An object that adopts the UICollectionViewDataSource protocol is responsible for providing the data and views required by a collection view.", so let's add those instance methods (many are marked as required). You can copy and paste the instance method declarations directly into your code, or simply start to type the method name and use Xcode's autocomplete functionality.

Add (or update) the following methods to the PicturesViewController.m file, before the closing @end line:

- (NSInteger)numberOfSectionsInCollectionView:(UICollectionView *)collectionView
{
    return 1;
}

- (NSInteger)collectionView:(UICollectionView *)collectionView numberOfItemsInSection:(NSInteger)section
{
    return 20;
}

- (UICollectionViewCell *)collectionView:(UICollectionView *)collectionView cellForItemAtIndexPath:(NSIndexPath *)indexPath
{
    PicturesCollectionViewCell *cell = [collectionView dequeueReusableCellWithReuseIdentifier:@"pictureCell" forIndexPath:indexPath];
    
    return cell;
}


At this point, you should be able to build and run the program, but you'll just see a blank (black) screen. The following steps are adjustments I made to better fit my images which are 188px x 188px. Click back on the MainStoryboard.storyboard, and click on Pictures Collection View Cell. Resize the cell to 188px x 188px.
view cell resize

I ensured the Image View was the same dimensions.
image view resize

To see the image layout in the simulator, click on the Attributes inspector and set the background for the Image view to white. Click on Collection View and click on size inspector, set the min spacing to 0. The above adjustments yield the following layout if you build/run the project:
ipad white cells

Now, it's time to make an API call and populate the image cells asynchronously. I opened PicturesViewController.m and added the following import at the top of the file:


#import "AFJSONRequestOperation.h"


I then added the following code to the viewDidLoad method after the line: [super viewDidLoad];


- (void)viewDidLoad
{
    [super viewDidLoad];
	// Do any additional setup after loading the view, typically from a nib.
        
    // URL/Request
    NSString *imagesUrl = [NSString stringWithFormat:@"http://pics.ericlondon.com/images.json"];
    NSURL *url = [NSURL URLWithString:imagesUrl];
    NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:[url standardizedURL]];
    
    // request parameters
    [request setHTTPMethod:@"GET"];
    [request setValue:@"application/x-www-form-urlencoded; charset=utf-8" forHTTPHeaderField:@"Content-Type"];
    
    // AF json request
    AFJSONRequestOperation *operation = [AFJSONRequestOperation JSONRequestOperationWithRequest:request
        success:^(NSURLRequest *request, NSHTTPURLResponse *response, id JSON) {
            
            [self.collectionView performBatchUpdates:^{
                
                NSArray *visibleCells = [self.collectionView visibleCells];
                
                for (NSInteger i=0; i <[visibleCells count]; i++) {
                    
                    PicturesCollectionViewCell *cell = visibleCells[i];
                    
                    NSString *thumbPath = [NSString stringWithFormat:@"http://pics.ericlondon.com%@", JSON[i][@"thumb_path"]];
                    NSURL *thumbPathURL = [NSURL URLWithString:thumbPath];
                    NSData *imageData = [NSData dataWithContentsOfURL:thumbPathURL];
                    UIImage *image = [[UIImage alloc] initWithData:imageData];
                    cell.pictureImageView.image = image;
                    
                }
                
            } completion:nil];            
            
        }
        failure:^(NSURLRequest *request, NSHTTPURLResponse *response, NSError *error, id JSON) {
            NSLog(@"ERROR: %@", error);
        }
    ];
    
    [operation start];
    
}


The above code makes an asynchronous JSON GET request to the Rails API which returns a JSON array containing image paths. The success block uses an objc block to update the image cells..
ipad with images

Next steps: sell app in Apple AppStore :)

Source code on GitHub
Please note: the content on this page orginates from ericlondon.com.
Here's a Ruby script I wrote that uses Selenium to authenticate and crawl railscasts.com, and Curb to download every RailsCasts Pro video. This assumes you have a paid RailsCasts account (http://railscasts.com/pro), dowit.


#!/usr/bin/env ruby

require 'selenium-webdriver'
require 'set'
require 'curb'

# define github credentials
github_email = 'YOUR GITHUB EMAIL ADDRESS'
github_password = 'YOUR GITHUB PASSWORD'

# define download dir, and get a list of existing files
download_dir = './downloads'
Dir.mkdir download_dir
Dir.chdir download_dir
existing_files = Dir.entries download_dir

# create new webdriver
driver = Selenium::WebDriver.for :firefox

# log into railscasts via github
driver.navigate.to 'http://railscasts.com/login'
driver.find_element(:id, 'login_field').send_keys github_email
element = driver.find_element(:id, 'password')
element.send_keys github_password
element.submit

# setup variables to contain navigation and episode links
nav_links_unscanned = ['http://railscasts.com/?page=1&view=list']
nav_links_scanned = []
episode_links = Set.new

# get a unique list of episode links
while !nav_links_unscanned.empty?
  link = nav_links_unscanned.shift
  nav_links_scanned << link

  driver.navigate.to link
  a_tags = driver.find_elements(:tag_name, 'a')
  a_tags.each do |a|
    # check for episode link
    if a[:href] =~ /.*\/episodes\/[0-9]+.*(?<!view=comments)$/
      episode_links << a[:href]
    # check for navigation link
    elsif a[:href] =~ /page.*view=list/ && !nav_links_unscanned.include?(a[:href]) && !nav_links_scanned.include?(a[:href])
      nav_links_unscanned << a[:href]
    end
  end

end

# loop through episode links and download movies
episode_links.each do |link|
  driver.navigate.to link

  # get movie link
  e = driver.find_element(:link_text, 'mp4')

  # download file
  file_name = e[:href].split('/').last
  if !existing_files.include?(file_name)
    existing_files << file_name
    puts "Downloading: #{e[:href]}\n"
    curld = Curl.get(e[:href])
    File.open(file_name, 'w') {|f| f.write(curld.body_str) }
  end

end

driver.quit


Source code on GitHub