Saturday, December 29, 2007

Automatic Asset Minimization and Packaging with Rails 2.0.x

With the recent release of Rails 2.0, many of us are reviewing our approaches to common problems. Many new features have been added to Rails, and some old tricks are either no longer necessary or no longer work.

I am developing a project with Rails 2.0 and am getting close to putting it into production. A recurring issue for today's web developers is that of asset packaging, or the combination of multiple site assets into a single file. Specifically, we're talking about Javascript and CSS.

A given "Web 2.0" (a term I wish had recently been found dead in a cramped apartment in Brooklyn) site might have a half dozen Javascript or CSS files to deliver to a user, and web browsers are not all that efficient at retrieving them. Each one requires a separate TCP connection to the server, and many browsers are only capable of getting two of these files concurrently. This means delays for your users.

In Rails 2.0 (and previously in Edge Rails), it's possible to combine multiple Javascript and CSS files using the javascript_include_tag and stylesheet_link_tag functions in your html.erb files; simply add :cache => true to the parameters like this:


<%= javascript_include_tag 'prototype', 'effects', :cache => true %>
<%= stylesheet_link_tag 'main', 'shop', 'form', :cache => true %>


With :cache => true and when running in your production environment, Rails will automatically combine your Javascript and CSS assets into single files (all.js and all.css, respectively) and significantly reduce your site's load time.

However, this really only solves part of the problem. A common technique used to further improve site performance is to compress Javascript and CSS by removing unnecessary whitespace and comments. I am not sure why this wasn't included as part of Rails' built-in caching features, but it seemed to me it should be easy to add.

Turns out I was mostly right. Google "javascript minimization" (or minification) and you'll see it's a pretty hot topic. The Asset Packager plugin from Scott Becker does this, as well as CSS compression, but is targeted at Rails 1.x and doesn't really make sense in the face of Rails 2.0.

So I set out to solve this problem in an elegant way for Rails 2.0. Asset Packager uses a Ruby script called jsmin.rb by Uladzislau Latynski which is based on jsmin.c by Douglas Crockford. The thing is, jsmin.rb is not a class or library, but rather a standalone executable that operates on stdin and stdout. Asset Pacakger actually forks a ruby shell process to do its Javascript minimization, and this seemed like folly if it could be done internal to Rails.

Accordingly, I modified jsmin.rb to operate as a singleton class and with a class method you could pass Javascript data to. Then it was simply a matter of monkey patching this function into ActionView::Helpers::AssetTagHelper, home of javascript_include_tag and stylesheet_link_tag.

I also wanted to add in CSS compression, which turned out to be easy. The javascript_include_tag and stylesheet_link_tag functions both use the same underlying functions to package their assets, so it was a simple case of replacing them with equivalents that do compression appropriately, based on whether we are dealing with CSS or JS.

config/initializers/javascript_minimization.rb:

module ActionView
module Helpers
module AssetTagHelper
require 'jsminlib'

def compress_css(source)
source.gsub!(/\s+/, " ") # collapse space
source.gsub!(/\/\*(.*?)\*\/ /, "") # remove comments
source.gsub!(/\} /, "}\n") # add line breaks
source.gsub!(/\n$/, "") # remove last break
source.gsub!(/ \{ /, " {") # trim inside brackets
source.gsub!(/; \}/, "}") # trim inside brackets
end

def get_file_contents(filename)
contents = File.read(filename)
if filename =~ /\.js$/
JSMin.minimize(contents)
elsif filename =~ /\.css$/
compress_css(contents)
end
end

def join_asset_file_contents(paths)
paths.collect { |path|
get_file_contents(File.join(ASSETS_DIR, path.split("?").first)) }.join("\n\n")
end

end
end
end



By simply modifying join_asset_file_contents to use our new function get_file_contents instead of File.read, we quickly get to the heart of the matter. CSS files get compress_css run on them, while Javascript files get JSMin.minimize run on them. Your :cache => true Javascript and CSS assets will now be gloriously combined and compressed!

Note that the above monkey patch requires jsminlib.rb, which you can download here. It is just a modified version of the original jsmin.rb, and you will want to put it into your Rails lib directory.

A good next step would be to further enhance get_file_contents to do Javascript obfuscation, which allows for the replacement of variable names and thus even further compression; it also tends to make Javascript code nearly incomprehensible and thus harder to steal, which may be desirable for some developers. I haven't found any native Ruby ways to do this yet, but it seems to me that this would be a good place for a C extension (or similar), and that this should all be put into a tiny and lightweight plugin.

I'm always amazed at how easy it is to bend Rails (and Ruby) to one's will, and in this case it's really quite elegant and straightforward. I'd love to hear your ideas about how to take this idea forward, potentially even including it in Rails itself.

Download the files here:

9 comments:

Erik said...

Thanks for writing this up. I had some trouble with an "undefined method 'options' for []:Array" in mongrel.rb. It turns out the downloadable version of your javascript_minimization.rb requires jsmin instead of jsminlib. Once I changed that it worked fine.

Dave Troy said...

Erik -- you're right. I will fix this now. Thanks for the heads up!

Simon said...

Thanks for this, just what I was looking for. I know it's not your method (you took it out of Asset Packager it seems) but you need to put "source" as the last line of compress_css -- gsub! returns nil if it doesn't do anything, so the whole function returns nil if you don't have a match for the last regex.

Dave Troy said...

Simon -- excellent point. You're right, will add it now!

subimage said...

Great article! Any stats on how much is saved by minifying + gzipping as opposed to gzipping alone?

This would be a good thing to put into rails core!

SmartFlix / HeavyInk said...

Both of the links to the files are broken.

Would love to give this a try!

bendtheblock said...

Great job, this is exactly what I'm looking for but the file links appear to be broken? Is this the case? Thanks.

lib/jsminlib.rb
config/initializers/javascript_minimization.rb

kylejginavan said...

nice work. this was exactly what i was looking for. i agree with you that by using ruby and rails elegant solutions easy solutions is usually the case.

i'm unable to download jsminlib.rb? would you please update your link?

cheers!

Dave Troy said...

The link to the source files has been fixed!