flouri.sh

If you've ever suffered from duplicate table aliasing problems in rails, there's a new feature in 2.2 that will help some of these situations go away. This seems to come up mostly in named_scope, where you want to define a list of scopes that can work by themselves or with any other scopes. This often means having identical or similar joins in multiple scopes. Take this example:

class User
  has_one :profile
  named_scope :male, {
    :join => "INNER JOIN profiles ON profiles.user_id = users.id",
    :conditions => "profiles.gender = 'male'" 
  }
  named_scope :recently_updated, {
    :join => "INNER JOIN profiles ON profiles.user_id = users.id",
    :conditions => ["profiles.updated_at > ?", 1.week.ago]	
  }
  named_scope :admin, {
    :join => "INNER JOIN profiles ON profiles.user_id = users.id AND profiles.admin = 1
              INNER JOIN emails ON emails.profile_id = profiles.id"
  }
  named_scope :with_profiles, {
    :join => :profile
  }
end

Before 2.2, calling User.male.recently_updated results in a table aliasing problem, because rails joins in the profiles table twice. Three features in 2.2 make this better:

• :join with hash/symbol syntax merges hashes when combining scope (Andrew White)
• :join now can take an array of strings (David Stevenson & Joseph Palermo)
• string identical joins are uniqued when combining scopes (David Stevenson & Joseph Palermo).

After 2.2, you can call User.male.recently_updated because there are two string identical joins combined in different scopes. You can't call User.male.admin without making some modifications, because the two joins involved are not string identical. Here's how I'd modify the :admin scope:

named_scope :admin, {
  :join => ["INNER JOIN profiles ON profiles.user_id = users.id",
            "INNER JOIN emails ON emails.profile_id = profiles.id"],
  :conditions => "profiles.admin = 1"
}

By using the array of strings :join syntax, I can let rails know that those are two separate joins, each of which can be merged if an identical join comes up in another scope. I also moved the extra join condition (profiles.admin = 1)to :conditions, so that the INNER JOIN statement would not have any specific logic for that particular named_scope in it.

I still can't call User.male.with_profiles because the string representation of the join from with_profiles probably doesn't match the join from male. It will be off due to the way rails generates join using different whitespace and escape characters than I originally wrote. This can be easily fixed by copying the exact string rails generates from :join => :profileand pasting it into male.

Developers finally set up master repo for acts_as_solr

After a long time of no development, a new acts_as_solr plugin has emerged on GitHub. Luke Francl & Mathias Meyer have adding a lot to the old plugin and created a new "master". Mathias has merged several branches on github including changes from David Palm, kengruven, and myself. This is great news for people running/modifying acts_as_solr, as Mathias is a real person who will actually respond to your pull requests. Some new features include:

• SOLR 1.3 installed
• Ruby 1.9 support
• LibXML 0.7+ support
• Store types as string not text fields in the index
• JVM options in solr.yml
• Completely rewritten test suite
• New solr:reindex task that automatically finds and indexes your solr models

I've been using query reviewer exensively at pivotal, and we've noticed that it is ridiculously slow when you have a lot of queries. This is due to the large number of partials rendered (5 partials) for each query in the reviewer box.

A solution to this problem, that also brings a huge advantage, was to report aggregate statistics for near-identical queries. Similar to way mysqlslowdump groups near-identical queries, query_reviewer now also reports only 1 line for X identical queries (identical means the same structure and same stack trace). Consequently, it's a lot easier to find places where you're missing includes . Here's what it looks like now:

The time reported for a group is the total time of all the queries in that group. In addition, I add up all the times for all the queries and report that number in the header. From this, I've noticed that a lot of time is spent in rails outside of waiting for mysql to return result. I wonder where that time goes...

query_reviewer can be downloaded from github!

Thanks to a nice suggestion by the community which I pretty much used verbatim. Still works with rails 2.0 and 1.2.3. Check out the project homepage.

http://code.google.com/p/query-reviewer/

Kevin Hall and I have released a new version of the query_reviewer plugin. You should start by looking at my first post, to see what the basic premise is before reading this article. The single largest improvment is the ability to analyze the database requests of AJAX requests, which is accomplished by piggy backing javascript or HTML into the ends of AJAX responses. Here's what it currently looks like, analyzing my project on a page that does lots of database requests:

The improvements are:

• View query analysis for AJAX requests
• Take into account the duration of a query (if production data)
• Show PROFILE information from mysql
• Warnings for long key lengths (which can be bad even when you hit an index)

Lots of information is now available for a single slow query:

I'm really excited that this plugin is getting attention, and welcome feedback, suggestions, and bug reports on the Google Code project homepage.

To install it, simply visit the project homepage.

Things started going very badly for flouri.sh when my home server wouldn't boot up. I removed almost all the hardware, but could not even get a POST screen or any beeps. I even reseated the RAM and CPU, but no luck. Finally, I grabbed an old media PC shuttle cube and attempted the craziest idea:

The 4 x RAID5 disks are actually being powered by my old server, but the PCI raid card is inserted into my old media PC. The shuttle doesn't have enough power or room to run all those disks so I'm using the old server as a lifeboat with 4 IDE cables running between them!

Using ActiveRecord can be tricky, here's how

Data migrations are a major headache in every rails project I've worked on. Developers typically write straight SQL migrations which take much longer to create and test, or they use ActiveRecord and run into problems. I recent wrote a data migration which took 7 existing tables and compressed them into 4. Some of the new 4 had the same names as the existing models, so I finally figured out how to do this safely with ActiveRecord. Here's my advice:

• When changing the schema of an existing set of tables, create new tables and then rename them.
• Include active record fragments inside the migration class at the top of your migrations.
• Put your data migrations inside transactions.
• Make your data migrations completely reversible (which is a lot easier when you follow the first rule)

Here's an example of an ActiveRecord class fragment at the top of a migration:

class MarkPrimaryBits < ActiveRecord::Migration
  class LessonVersion < ActiveRecord::Base
    belongs_to :lesson
  end

  class Lesson < ActiveRecord::Base
    has_many :lesson_versions
  end
 
  def self.up
    ...

By putting the Lesson and LessonVersion classes at the top of the migration, I'm allowing myself to use those classes inside my migration and be completely independent of any changes made to the real model from then on (including the deletion of the class itself). Furthermore, I can have another migration which uses those same class names with completely different meanings and they won't conflict with each other.

How to build custom methods on associations that aren't slow

The "has_many do" syntax has been widely adopted in rails, but I often see it going wrong. Consider the following association extension:

has_many :versions do
  def primary
    find(:first, :conditions => {:primary => true})
  end
end

What's wrong with this code is that the finder has to execute a database request every time it's invoked. True, rails has a query cache, but it results in more database requests then one should need. We can make it better by caching the result in an instance variable attached to the association:

has_many :versions do
  def primary
    @primary ||= find(:first, :conditions => {:primary => true})
  end
end

But what if you already have the association loaded? Inside the block, you can access various methods of the AssociationCollection and AssociationProxy classes. Notable methods are the following:

• proxy_owner - the module that contains the association
• proxy_reflection - the Reflection object that contains the association options (FK, :dependent, etc)
• proxy_target - the cached association data, if the association has been loaded
• loaded? - returns true if the association has been loaded
• reset - delete the cached association data and forget it has been loaded

Using these methods, we can rewrite our method to be the most efficient possible, by making use of loaded association data when present:

has_many :versions do
  def primary
    if loaded?
      @primary ||= proxy_target.detect { |ver| ver.primary? }
    else
      @primary ||= find(:first, :conditions => {:primary => true})
    end
  end
end

Now our method is guarenteed to make only 1 database call, no matter how many times invoked. It also will make zero database calls if the entire association has already been loaded. The only downside is that "how to determine if primary" logic has to be written once in rails_sql and once in pure ruby.

query_analyzer on steriods!

Features

QueryReviewer is an advanced SQL query analyzer. It accomplishes the following goals:

• View all EXPLAIN output for all SELECT queries to generate a page
• Rate a page's SQL usage into one of three categories: OK, WARNING, CRITICAL
• Attach meaningful warnings to individual queries
• Find out where the query was executed with a stack trace
• Display advanced interactive summary on page

It accomplishes this by injecting an absolutely positioned div into your HTML markup before it's sent out of rails. View injection can be turned off, if necessary and replaced with <%= query_review_output %> somewhere in your view.

Screenshot

Project homepage

http://code.google.com/p/query-reviewer/

Installing

svn export http://query-reviewer.googlecode.com/svn/trunk/ vendor/plugins/query_reviewer

Then optionally run "rake query_reviewer:setup" and edit config/query_reviewer.yml to change the default settings.

All the flickr functionality with better iPhoto integration!

I just finished a conversion of my existing flickr ajax mephisto plugin to use mac.com web galleries! I finally ended up paying for mac.com, because of the awesome integration with everything.

Features:

• Pull photos from a specific album or your entire gallery
• Uses AJAX to pull feed data, so it won't bottleneck your main page rendering
• Randomly chooses X photos to display every time
• A variety of supported images sizes: square, small, medium, and large
• Images automatically link to Apple's awesome web gallery interface

Issues (depending on size of gallery, mine has 400 images at the time):

• The RSS feed takes between 2 and 10 seconds to deliver from apple.
• The downloaded XML file takes 2 to 5 seconds process.

Usage

How I used it in my site:

<p>Pictures:</p>
{% macgallery feed: http://gallery.mac.com/dsboulder/?webdav-method=truthget&feedfmt=photocastrss  count: 4  format: square  width: 75  height: 75 %} 
{% endmacgallery %}

Supported options to the "macgallery" liquid tag:

• feed: REQUIRED. The url of the RSS feed to your gallery, or to a specific album.
• count: OPTIONAL (default: 6). The number of images to display.
• format: OPTIONAL (default: square). Also valid are "small", "medium", and "large".
• width/height: OPTIONAL (default: none). If specified, will add width and height attributes to the IMG tag. Useful for resizing the square images from their 160x160 default size.

Where to get the plugin

After I wrote the article on hacking UV into mephisto, I made it an actual plugin. The only trick was inserting a fake codepress.rb file since mephisto requires "codepress" in 2 places.

To get up and running:

• Delete the existing filtered_column_code_macro plugin.
• ./script/plugin install http://wush.net/svn/public/plugins/mephisto/uv_code_macro
• Copy over UV's xhtml theme CSS files. See using UV as a library.
  You'll have to run something like Uv.copy_files "xhtml", "public/stylesheets" from the console
• Include one or more of the new stylesheets in your layout.
• Use <filter:code lang="XXXX" theme="XXXX"> tags to highlight your code!

I'm tired of waiting for my blog to load because of a slow flickr feed request! I designed this new plugin by copying the flickr photostream plugin, but making it actually load the pictures and the feed through an immediate AJAX request. This tremendously increased the performance of my mephisto pages!

The plugin defines a flickr controller, which accepts the AJAX request. Unfortunately, you can't change how the photos are laid out anymore (I'm not setting up a liquid context for this ajax response). Of course, if you don't like the result of the plugin, just change it to match what you want:

class FlickrController < ActionController::Base
  def index
    result = ""
    pics = find_pictures
    pics.each do |pic|
      result << "<a href='#{pic.link}'><img src='#{pic.send(params[:format].to_sym)}' alt='#{pic.title}'></a>"
    end

    render :inline => result
  end
end

How to install and use

• Uninstall the flickr photostream plugin
• ./script/plugin install http://wush.net/svn/public/plugins/mephisto/mephisto_flickr_ajax
• Use the following tag in your liquid template:
  {{flickrajaxphotostream feed:<YOUR_FEED_URL> count:<NUMBER_IMAGES> format:<[square, small, etc]>}}
{{endflickrajaxphotostream}}
Originally posted on ELC's blog

Why the heck do you want to do that?

Views are very well integrated into the rails framework, but they're only typically rendered when an http request comes in. ActionMailer is the main exception, but what if you want to render a view for use in another backend application? These days, document fragments are being used everywhere, and often times I'll need rendered HTML for a use other than just sending it back to the requesting user.

A solution: instantiate a controller and view

Controllers are just objects, and so are views. We can instantiate a controller, instantiate a view, then point the view to the controller and we're ready to go. The only think we're missing is the session and the request objects, but not every view needs those. I used this once for updating a facebook profile using a backgroundrb worker:

class FakeView < ActionView::Base
  include SomeHelper
  include SomeOtherHelper
end

class FakeController < ActionController::Base
  def render_some_view
    action_view = FakeView.new(File.join(RAILS_ROOT, "app", "views"), {})
    action_view.instance_variable_set("@controller", self)
    markup = action_view.render(:partial => 'facebook/your_profile')    
  end
end

By subclasses ActionView::Base, you can mix helpers into the view class, making their methods available.

Another solution: use the test framework!

The rails TestProcess is the only place where views are rendered. If you really want to simulate a real experience when rendering a view, use the test process. First, you'll need an actual controller with an actual action you want to render in it, like this one:

class FacebookController
  before_filter :login_required
  
  def your_profile    
  end
end

Then, we create and instante a test as follows:

class FacebookTest
  include ActionController::TestProcess
  attr_reader :response
  
  def initialize
    require_dependency 'application' unless defined?(ApplicationController)
    @controller = UserScheduleEntryController.new
    @request    = ActionController::TestRequest.new
    @response   = ActionController::TestResponse.new
  end
  
  def render_your_profile(user)
    @controller.instance_variable_set("@user", user) # bypass login required
    get :your_profile
    @response
  end  
end

test = FacebookTest.new
test.render_your_profile(user)
markup = test.response.body

The require_dependency was something I threw in because backgroundrb didn't have some of the required classes loaded at that point, it may not be something you need in your application.

Ever since moving to San Francisco, I can't believe how fast internet access can be here! Check it out:

The sample above was taken at my house... middle of the day... using a cable modem with Comcast over 802.11n wireless!