Sunday, September 23, 2007

Twitter4R Announcements

First off I have been meaning to post an entry letting people know of a recently released Rails application written by Sergio Santos called TwitterNotes that is using the Twitter4R gem.

Thanks Sergio (and anyone else that worked on the project - sorry if I left you off).

Also Sergio emailed yesterday requesting message paging support with a suggested patch (thanks for contributing back - muchas gracias). This evening I released Twitter4R version 0.2.5 that incorporates the necessary code changes for this support.

What does this mean?

Previously you would have only been able to get the last 20 sent/received direct messages from the Twitter4R bindings doing something like the following:


messages = client.messages(:sent)

Now you can request specific pages like so:

messages = client.messages(:sent, :page => 2)

The same is true for the :received case.

To install the Twitter4R RubyGem simply run the following in your terminal/console:
$ sudo gem install twitter4r


Give the Rubyforge mirrors a few hours to sync if you are only getting Twitter4R v0.2.4.

Thanks and enjoy!

Friday, September 21, 2007

Custom Processors for ActiveWarehouse ETL

For anyone interested in extending ActiveWarehouse ETL's features with custom pre/post processors, I thought I would share this piece of code that I wrote in August for a personal project I am working on. The example should provide you with enough details for you to create your own custom processors.


# Written by Susan Potter under open source MIT license.
# August 12, 2007.

require 'net/ftp'

module ETL
module Processor
# Custom pre-processor to download files via FTP before beginning control process.
class FtpDownloaderProcessor < ETL::Processor::Processor
attr_reader :host
attr_reader :port
attr_reader :remote_dir
attr_reader :files
attr_reader :username
attr_reader :local_dir

# configuration options include:
# * host - hostname or IP address of FTP server (required)
# * port - port number for FTP server (default: 21)
# * remote_dir - remote path on FTP server (default: /)
# * files - list of files to download from FTP server (default: [])
# * username - username for FTP server authentication (default: anonymous)
# * password - password for FTP server authentication (default: nil)
# * local_dir - local output directory to save downloaded files (default: '')
#
# As an example you might write something like the following in your control process file:
# pre_process :ftp_downloader, {
# :host => 'ftp.sec.gov',
# :path => 'edgar/Feed/2007/QTR2',
# :files => ['20070402.nc.tar.gz', '20070403.nc.tar.gz', '20070404.nc.tar.gz',
# '20070405.nc.tar.gz', '20070406.nc.tar.gz'],
# :local_dir => '/data/sec/2007/04',
# }
# The above example will anonymously download via FTP the first week's worth of SEC filing feed data
# from the second quarter of 2007 and download the files to the local directory +/data/sec/2007/04+.
def initialize(control, configuration)
@host = configuration[:host]
@port = configuration[:port] || 21
@remote_dir = configuration[:remote_dir] || '/'
@files = configuration[:files] || []
@username = configuration[:username] || 'anonymous'
@password = configuration[:password]
@local_dir = configuration[:local_dir] || ''
end

def process
Net::FTP.open(@host) do |conn|
conn.connect(@host, @port)
conn.login(@username, @password)
remote_files = conn.chdir(@remote_dir)
@files.each do |f|
conn.gettextfile(remote_file(f), local_file(f))
end
end
end

private
attr_accessor :password

def local_file(name)
File.join(@local_dir, name)
end

def remote_file(name)
File.join(@remote_dir, name)
end
end
end
end

The key things to note from this is that you are at present required to:
  • define all your custom processors with in the ETL::Processor module
  • name your custom processor class in the form XXXXProcessor
  • need to extend (or really just adhere to the message interface of) ETL::Processor::Processor class defined in ActiveWarehouse ETL
  • define initialize taking two arguments (look above for guidance)
  • define a process method to do what you need do before or after the control process runs (for pre and post processors respectively)

Hope this helps someone customize ActiveWarehouse more easily, since the only bad thing I have found with ActiveWarehouse is lack of documentation.

Thursday, September 13, 2007

Separating Rails Layout Associations

I just realized I hadn't shared this code yet on this blog. I will also be including it in one of the metafusion subprojects (coming soon). I've been using it when needed in my Rails projects on and off for the last several months.

The Problem

You have some controllers you include into your application from plugins or engines and you want to associate a particular layout to the plugin/engine controller from within your application (without changing anything in the plugin or engine - of course!). There is not good way of doing this nicely in Rails presently as each controller usually defines controller-wide layouts within its definition.

The Solution



[finsignia/paths.rb]

# See:
# * Finsignia::Paths

# Contains helper methods related to paths and resolving modules and
# classes from paths. Provides for helpful mixin for various applications.
module Finsignia::Paths

def self.included(base)
base.extend ClassMethods
end

# Contains class methods for Finsignia::Paths mixin
module ClassMethods
@@element_postfixes = {:model => '', :controller => 'Controller'}

def resolve_model(path)
resolve_element :model, path
end

def resolve_controller(path)
resolve_element :controller, path
end

def normalize_module_name(path)
list = path.split('_')
list.collect do |item|
item.capitalize
end.join
end

# Resolves path to a type of Rails element
# (e.g. model, controller, etc.).
#
# Path refers to 'internal' path, NOT require path:
# 'users/users' #=> internal path
# 'users/users_controller #=> require path
def resolve_element(type, path)
first, rest = path.split('/')
mod = ObjectSpace.const_get(normalize_module_name(first))
while rest
first, rest = rest.split('/')

unless rest
mod = mod.const_get(normalize_module_name("#{first}_#{@@element_postfixes[type].downcase}"))
else
mod = mod.const_get(normalize_module_name(first))
end
# I doubt this will ever get this far as per expeceted behavior of ObjectSpace and Module...so commented it out following unreachable line.
#raise NameError.new("#{type} constant not found for internal path #{path}") unless mod
return mod unless rest
end
end

# Separating this so we can stub this method out in specifications.
def require_controller(path)
require("#{path}_controller") unless "test" == ENV["RAILS_ENV"]
end

# Separating this so we can stub this method out in specifications.
def require_model(path)
require(path) unless "test" == ENV["RAILS_ENV"]
end
end
end

[finsignia/layouts.rb]

# See:
# * Finsignia::Layouts
# * Finsignia::LayoutsError

# Raised when an exceptional condition arises in the Layouts
# mapping process.
class Finsignia::LayoutsError < Exception; end

# Provides a closure and declarative way to define layout
# mappings for an application that utilize controllers, views,
# etc. from plugins.
#
# The closure approach simulates the Rails Routing approach
# closely, like:
# require 'application'
#
# Finsignia::Layouts.map do |map|
# map.connect 'users/sessions', 'layout_name'
# end
#
# The declarative approach looks like the following:
# require 'application'
# include Finsignia
# Layouts.map 'users/sessions', 'layout_name'
#
module Finsignia::Layouts
class << self
def connect(controller_path, layout)
require_controller(controller_path)
# resolve controller class and call .layout(layout) on it.
controller = resolve_controller(controller_path)
controller.layout(layout)
end

def map(&block)
yield self if block_given?
end
end

# private
include Finsignia::Paths
end


So basically in a file like config/layouts.rb of our application we have something like:


Finsignia::Layouts.map do |map|
map.connect 'users/sessions', 'session'
map.connect 'users/users', 'users'
map.connect 'accounts/transaction', 'account'
end


Then include the config/layouts.rb file in config/environment.rb and you have separated your concerns relatively nicely and easily.

When I release metafusion-rails my plan is that instead of installing the finsignia/paths.rb and finsignia/layouts.rb in the lib directory you would be able to do something like the following at the end of your config/environment.rb file:

gem('metafusion-rails', '=MFR_VERSION')
require 'metafusion/rails'

Finsignia::Layouts.map do |map|
map.connect 'namespace/resource', 'layout'
# etc....
end

Alternatively you could still keep the layouts.rb file if your environment.rb is getting large and keep the layouts.rb include in your environment.rb.

Tuesday, September 11, 2007

Model-Conductor-Controller fix

The New Bamboo blog talked about Presenters & Conductors last month. The demonstration Rails application was broken, so the enclosed link to a zip file contains the fixes I made to get the application running as expected.

The offending code was in lib/action_conductor/lib/action_conductor/errors.rb:



module ActiveRecord
class Errors
def add_conductor_errors_for(record, mapping)
return if record.valid?
record.errors.each do |attribute, message|
self.add(mapping[attribute.to_sym], message)
end
end
end
end


The fixed code changed one line as so:


module ActiveRecord
class Errors
def add_conductor_errors_for(record, mapping)
return if record.valid? or record.new_record?
record.errors.each do |attribute, message|
self.add(mapping[attribute.to_sym], message)
end
end
end
end


The fixed zip file can be found at my
4shared account and also fixes the minor view errors.