Working with Delayed Job

Delayed Job is great for offloading processes out of the way for users.

For Heraldr, we needed to be able to send a lot of emails from the site without bogging down the performance.

Enter Delayed Job. It uses a table in the db to store jobs in a que to be processed in the background.

This is helpful because the data is stored in the db and will not be lost if there is a server hiccup (as is a danger with other rails background processes).

We looked at different options, but Delayed Job worked best for what we needed.

We also recommend using the plugin instead of the gem. It worked a little smoother in dev mode.

How to use Delayed Job

Install the plugin from git hub into your plugins folder.

ruby script/plugin install git://github.com/tobi/delayed_job.git

Once you have installed the plugin you need to create a Delayed Job Migration and add the jobs table to the database.

ruby script/generate migration create_table_for_delayed_job

#in the migration file
class CreateTableForDelayedJob < ActiveRecord::Migration
  def self.up
    create_table :delayed_jobs, :force => true do |t|
      t.integer  :priority, :default => 0
      t.integer  :attempts, :default => 0
      t.text     :handler
      t.string   :last_error
      t.datetime :run_at
      t.datetime :locked_at
      t.datetime :failed_at
      t.string   :locked_by
      t.timestamps
    end
  end

  def self.down
    drop_table :delayed_jobs
  end
end

rake db:migrate

Now that you have a delayed_job table, you have options on how to add jobs to the que.

For the easiest way, you can simply add:

Model.send_later(:method, object, second object if necessary)

This will use the plugin defaults in the delayed_job folder for the amount of time delayed, etc.

You can also create a custom class to handle the processing instead of using send_later() like:

  class NewsletterJob < Struct.new(:text, :emails)
    def perform
      emails.each { |e| NewsletterMailer.deliver_text_to_email(text, e) }
    end    
  end  
  
  #Use this in where you need to call the Delayed Job task
  Delayed::Job.enqueue NewsletterJob.new('lorem ipsum...', Customers.find(:all).collect(&:email))

This example is from the readme file, which does a pretty good job of explaining everything.

The class just acts as a custom wrapper to enque the tasks you need to run in the background.

With the latest version, you can have multiple Delayed_job workers running as long as they are separate (and you have the system rescources).

Running the worker

In dev mode, it is easy to run:

rake jobs:work

Delayed Job will then cycle through the table sending one job at a time.

rake jobs:clear

Will remove all jobs from the db.

In production, you can write a script to start and stop the worker upon deploying.

After a few tries and a couple different gems and methods, we found Amit Mathur's script used with the daemon-spawn gem to be most effective.

Just create a delayed_job file in the /scripts folder and copy the code there.

Be sure to add github to your gem sources if you have not already for the daemon-spawn gem.

sudo gem sources -a http://gems.github.com
sudo gem install alexvollmer-daemon-spawn

daemon-spawn proved to be more stable and easier to work with then the daemon gem.

Amit also has a very good Delayed Job tutorial on Delayed Job that is worth reading.

Amit also quotes CollectiveIdea's Cap recipe which worked well.

Just add it to the end of your deploy.rb file.

# add this to config/deploy.rb
namespace :delayed_job do
  desc "Start delayed_job process" 
  task :start, :roles => :app do
    run "cd #{current_path}; script/delayed_job start #{rails_env}" 
  end

  desc "Stop delayed_job process" 
  task :stop, :roles => :app do
    run "cd #{current_path}; script/delayed_job stop #{rails_env}" 
  end

  desc "Restart delayed_job process" 
  task :restart, :roles => :app do
    run "cd #{current_path}; script/delayed_job restart #{rails_env}" 
  end
end

after "deploy:start", "delayed_job:start" 
after "deploy:stop", "delayed_job:stop" 
after "deploy:restart", "delayed_job:restart"

The most difficult part is setting up the worker for production. Delayed Job pretty much works out of the box in dev mode.

Be sure the rake command works first in production mode by entering the rake command on the server's command line to confirm that Delayed Job can connect to the db.