>> JSON({:a => "\360\237\230\204"}.to_json)
=> {"a"=>"\357\230\204"}

The problem is related to the Rails internals and its JSON escaping process. After digging around internet i’ve found that there’re few posts that explain the bug and also provide few solutions.

• Github rails issue #3727
• Stackoverflow question

Solution

The issue gets fixed by placing the following content into config/initializers/active_support_encoding.rb file:

module ActiveSupport::JSON::Encoding
  class << self
    def escape(string)
      if string.respond_to?(:force_encoding)
        string = string.encode(::Encoding::UTF_8, :undef => :replace).force_encoding(::Encoding::BINARY)
      end
      json = string.gsub(escape_regex) { |s| ESCAPED_CHARS[s] }
      json = %("#{json}")
      json.force_encoding(::Encoding::UTF_8) if json.respond_to?(:force_encoding)
      json
    end
  end
end

If you’re rolling deployments with capistrano, it will insert the REVISION file under the app’s root dir with git sha or svn revision or whatever tag based on SCM of your choice.

Here is the simple rack middleware that injects that revision as a ‘X-REVISION’ header in responses.

module Rack
  class Revision
    @@revision = nil
    File = ::File
 
    def initialize(app, &block)
      @app   = app
      @block = block
      @file  = File.join(Dir.pwd, 'REVISION')
    end
 
    def call(env)
      status, headers, body = @app.call(env)
      headers['X-Revision'] = revision
      [status, headers, body]
    end
 
    protected
 
    def revision
      @@revision ||= read_revision
    end
 
    def read_revision
      File.exists?(@file) ? File.read(@file).strip : 'UNDEFINED'
    end
  end
end

For instance, just put that file as lib/rack/revision.rb and change your config.ru file:

require ::File.expand_path('../config/environment',  __FILE__)
use Rack::Revision
run YOUR_APP::Application

You’ll need to restart the app to apply the changes.

To test if it works just run:

curl -i -X HEAD "http://YOUR_WEBSITE/"

Sample output would be:

ETag: "8d6228d86642025c31e3d54e9a67b14b"
Cache-Control: max-age=0, private, must-revalidate
X-Runtime: 0.001491
X-Rack-Cache: miss
X-Revision: f8c51630843898e88848261dd5ebac3fdebc5e48

The issue – capistrano loads default configuration and then executes your staging task and overrides previously defined variables. Default environment before executing stage task is set to :production. So unless you define :rails_env, :unicorn_env, :app_env in your stage config it will use a wrong environment.

Lets take a look at sample deploy.rb file:

set :stages, %w(production staging)
set :default_stage, "staging"
require 'capistrano/ext/multistage'

You’ll need to add config/deploy/staging.rb and production.rb files:

set :domain,      "YOUR_HOST"
set :rails_env,   "staging"
set :unicorn_env, "staging"
set :app_env,     "staging"
 
role :web, domain
role :app, domain
role :db,  domain, :primary => true
 
set :deploy_to,   "/home/#{user}/#{application}/#{fetch :app_env}"
set :current_path, File.join(deploy_to, current_dir)

This should fix the problem with wrong rails env being passed to unicorn server.

Error: undefined method `include?' for nil:NilClass
Please report this bug:
    https://github.com/mxcl/homebrew/wiki/checklist-before-filing-a-new-issue
/usr/local/Library/Homebrew/formula.rb:417:in `canonical_name'
/usr/local/Library/Homebrew/formula.rb:449:in `factory'
/usr/local/Library/Contributions/examples/brew-upgrade.rb:7
/usr/local/Library/Contributions/examples/brew-upgrade.rb:7:in `map'
/usr/local/Library/Contributions/examples/brew-upgrade.rb:7
/usr/local/bin/brew:47:in `require'
/usr/local/bin/brew:47:in `require?'
/usr/local/bin/brew:80

This happens because of old brew-update command. Pretty annoying, so to fix it just run the following script:

cd `brew --prefix`/Library/Contributions/examples
git clean -n
git clean -f

I use capistrano for all my apps and find it very useful for quick and handy deployments. After switch to unicorn i’ve been copying-and-pasting the same code over and over again, so i decided to move it to a separate gem.

Check out code on github – capistrano-unicorn.

Usage

Assuming that you already have your capistrano script ready, add gem to the Gemfile:

group :development do
  gem 'capistrano-unicorn'
end

Then, add requirement to the deploy.rb file (after all hooks):

require 'capistrano-unicorn'

Configuration

All unicorn configs should be placed under APP_ROOT/config/unicorn/ENVIRONMENT.rb
So, for each environment there should be a separate file.

Test if that worked:

cap unicorn:start
cap unicorn:stop
cap unicorn:reload

For more information check out github repository – capistrano-unicorn.

It is still under development, but basic functionality works fine.

Capistrano versions: >= 2.5

Configuration

First, you need to define application environment:

# Define server-side rails environment
set :rails_env, "production"
 
# Primary deployment location
set :deploy_to, "/home/#{user}/#{application}"
 
# Place where all backups will be dumped
set :backup_to, "/home/#{user}/#{application}/backups"

Also, add this function. It allows capistrano to check if remote file exists:

def remote_file_exists?(full_path)
  'true' ==  capture("if [ -e #{full_path} ]; then echo 'true'; fi").strip
end

Now, we need to add a backup task:

namespace :utils do
  desc 'Backup database before deploy'
  task :backup, :roles => :db, :only => {:primary => true} do
    run "mkdir -p #{backup_to}" # Create a backup folder unless exists
 
    # Primary backup filename
    filename = "#{backup_to}/#{application}_predeploy_#{Time.now.strftime("%m%d%Y%H%I%S")}.sql.gz"
 
    # Check if we've got database config
    if remote_file_exists?("#{deploy_to}/current/config/database.yml")    
      text = capture("cat #{deploy_to}/current/config/database.yml")
      config = YAML::load(text)[rails_env]
 
      on_rollback { run "rm #{filename}" }
      run "mysqldump -u #{config['username']} -p #{config['database']} | gzip --best > #{filename}" do |ch, stream, out|
        ch.send_data "#{config['password']}\n" if out =~ /^Enter password:/
      end
    else
      logger.debug("[Backup] No configuration file was found.")
    end
  end
end

And finally, add the capistrano before hook:

before "deploy", "utils:backup"

Testing

To test if the task works, run:

cap utils:backup

On the server side you should see a backup file:

/home/USER/APP/backups/APP_predeploy_MMDDYYHHMMSS.sql.gz.

Stack:

• Debian / Ubuntu Server
• Postix
• Ruby 1.9
• Rails 3.0

Overview

You have an application where users get email notifications. And you want to allow them to reply directly to the email.
In order to do so, each email should have an unique (depends on situation) reply-to address. Usually its something like that:

P946d272cf7da4dd6b0cb613605bced65@yourdomain.com

This means that the mailserver you use should support dynamic/virtual email addresses and forwarding.

Postfix Configuration

First, you’ll need to install postfix (in not installed):

apt-get install postfix

Configuration should look like this:

# See /usr/share/postfix/main.cf.dist for a commented, more complete version

default_privs = apps

# Debian specific:  Specifying a file name will cause the first
# line of that file to be used as the name.  The Debian default
# is /etc/mailname.
#myorigin = ap

smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
biff = no

# appending .domain is the MUA's job.
append_dot_mydomain = no

# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h

readme_directory = no

# TLS parameters
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_use_tls=no
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

# See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for
# information on enabling SSL in the smtp client.

myhostname = YOUR_APP_HOSTNAME
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydomain = YOUR_APP_DOMAIN
mydestination = YOUR_APP_DOMAIN, localhost
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
relay_domain = localhost
recipient_canonical_maps = regexp:/etc/postfix/recipient_canonical

Last option recipient_canonical_maps allows you to define a dynamic email addresses and forward them to the specific system mailbox for processing.

Create a file /etc/postfix/recipient_canonical:

/^P[0-9abcdef]{1,}(M[0-9]{1,})?/ apps

This will add a virtual recipient addresses and forward messages to user apps.

NOTE: Regular expressions should be in POSIX format. For test you can use regextester.com

Email Aliases

After you added support for virtual addresses all mail will be delivered to the system user mailbox (apps). But, we need to drive all that traffic into our app. In order to do so we will have to setup mail piping directly into your application script.

Edit /etc/aliases:

apps: "| /home/apps/APP_NAME/current/script/email_receiver_script"

And rebuild the aliases db by running:

newaliases

Do not forget to restart postfix:

/etc/init.d/postfix restart

You can test out the email delivery. For errors check /var/log/mail.info

Mail Receiver Script

Since all mail will be forwarded directly to our mail receiver script via piping there are few things to consider:

• Email receiver should consume as less memory as possible.
• Email receiver should not load the whole application (because of item above).
• Email receiver should only validate and preprocess incoming messages and leave actual processing to another subsystem via queue.

Configuration

There are few ruby libraries that are well suited for this case:

• mail – Email processing, ruby 1.9.2 compatible (comparing to tmail which is not)
• redis – Simple key-value in-memory database.
• resque – Redis-backed library for creating background jobs, placing those jobs on multiple queues, and processing them.

Install gems:

gem install mail redis resque

Here is an example email receiver script:

#!/usr/bin/env ruby 

require 'rubygems'
require 'mail'
require 'redis'
require 'resque' 

class EmailReply
  @queue = :email_replies 

  def initialize(content)
    mail    = Mail.read_from_string(content)
    from    = mail.from.first
    to      = mail.to.first 

    if mail.multipart?
      part = mail.parts.select { |p| p.content_type =~ /text\/plain/ }.first rescue nil
      unless part.nil?
        message = part.body.decoded
      end
    else
      message = part.body.decoded
    end 

    unless message.nil?
      Resque.enqueue(EmailReply, from, to, message)
    end
  end
end 

EmailReply.new($stdin.read)

This script receives the mail message then tries to extract the plaintext body. If the email message is valid it adds it to the queue for future processing.

Mail Queue processing

After we put emails into the queue we’ll need to create a worker.

If you need to extract a reply from the body, use (mail_extract)[https://github.com/sosedoff/mail_extract]:

gem install mail_extract

Simple worker (resque job worker), extracted from one of the projects. (RAILS_ROOT/lib/email_reply.rb):

class InvalidReplyUUID    < StandardError ; end
class InvalidReplyUser    < StandardError ; end
class InvalidReplyProject < StandardError ; end
class InvalidReplyMessage < StandardError ; end 

class EmailReply
  @queue = :email_replies 

  def self.parse_email_uuid(str)
    if str =~ /^P[0-9abcdef]+(M[\d]+)?@/i
      parts = str.scan(/^P([0-9abcdef]+)(M([\d]+))?/).flatten
      project_uuid = parts.first
      message_id = parts.size == 3 ? parts.last : nil 

      result = {:project_uuid => project_uuid}
      result[:message_id] = message_id unless message_id.nil?
      result
    else
      raise InvalidReplyUUID, "Invalid UUID: #{str}"
    end
  end 

  def self.perform(from, to, body)
    user = User.find_by_email(from)
    if user.nil?
      raise InvalidReplyUser, "User with email = #{from} is not a member of the app."
    end 

    info = parse_email_uuid(to) 

    project = Project.find_by_uuid(info[:project_uuid])
    if project.nil?
      raise InvalidReplyProject, "Project with UUID = #{info[:project_uuid]} was not found."
    end 

    if info.key?(:message_id)
      message = project.messages.find_by_id(info[:message_id])
      if message.nil?
        raise InvalidReplyMessage, "Message with ID = #{info[:message_id]} was not found on project '#{project.name}'"
      end
    end 

    params = {
      :project  => project,
      :body     => MailExtract.new(body).body,
      :markup   => 'plain',
      :sent_via => 'email'
    }
    params[:message] = message unless message.nil? 

    message = user.messages.new(params)
    unless message.save
      raise RuntimeError, "Unable to save message. Errors: #{message.errors.inspect}"
    end
  end
end

NOTE: Its important that both mail receiver and worker are using the same queue.

Create a resque.rake in RAILS_ROOT/lib/tasks:

require 'resque/tasks'
task "resque:setup" => :environment

And fire it up:

rake resque:work QUEUE=email_replies

Source: https://github.com/ddollar/foreman

Installation

gem install foreman

Configuration

Lets assume you have a resque worker, which you execute with rake:

bundle exec rake resque:work QUEUE=* RAILS_ENV=production

Now, you will need to create a Procfile inside your rails root folder (Procfile):

worker: bundle exec rake resque:work QUEUE=* RAILS_ENV=production

Start your workers with foreman:

foreman start

Foreman will start all of the processes associated with your app and display stdout and stderr of each process. Processes are color-coded by type to make them easy to read. $PORT will be automatically assigned by foreman and made available to each underlying process.

Export to system

While foreman is great for running your application in development, there are some great existing tools for managing processes in production. Foreman can export to either upstart or standard unix init.

Exporting to upstart:

$ foreman export upstart /etc/init

[foreman export] writing: /etc/init/testapp.conf
[foreman export] writing: /etc/init/testapp-web.conf
[foreman export] writing: /etc/init/testapp-web-1.conf

[foreman export] writing: /etc/init/testapp-worker.conf
[foreman export] writing: /etc/init/testapp-worker-1.conf
[foreman export] writing: /etc/init/testapp-clock.conf

[foreman export] writing: /etc/init/testapp-clock-1.conf

Capistrano integration

Assuming you got your capistrano recipe working, here is a set of foreman tasks:

# Foreman tasks

namespace :foreman do
  desc 'Export the Procfile to Ubuntu upstart scripts'
  task :export, :roles => :app do

    run "cd #{release_path} && sudo bundle exec foreman export upstart /etc/init -a #{application} -u #{user} -l #{release_path}/log/foreman"

  end

  desc "Start the application services"
  task :start, :roles => :app do

    sudo "start #{application}"
  end

  desc "Stop the application services"

  task :stop, :roles => :app do
    sudo "stop #{application}"

  end

  desc "Restart the application services"
  task :restart, :roles => :app do

    run "sudo start #{application} || sudo restart #{application}"
  end
end

Just add deploy hooks and its ready to go:

after "deploy:update", "foreman:export"    # Export foreman scripts
after "deploy:update", "foreman:restart"   # Restart application scripts

Documentation

• Github repo – https://github.com/ddollar/foreman
• Blog post – http://blog.daviddollar.org/2011/05/06/introducing-foreman.html

Here is a Rack::Maintenance module that provides such functionality:

module Rack
  class Maintenance
    File = ::File
 
    def initialize(app, dir=nil, &block)
      @app = app
      @block = block
      @dir = dir || Dir.pwd
      @file = File.join(File.expand_path(@dir), '.maintenance')
    end
 
    def call(env)
      if File.exists?(@file)
        body = @block.nil? ? default_prompt(time_info) : @block.call(time_info)
        res = Response.new
        res.write(body)
        res.finish
      else
        @app.call(env)
      end
    end
 
    private
 
    def time_info
      t_since, t_until = [nil, nil]
      File.open(@file) do |f|
        t_since = f.mtime
        t_until = Time.parse(f.gets) rescue nil
        t_until = nil if t_until.kind_of?(Time) && t_until < Time.now
      end
      {:since => t_since, :until => t_until}
    end
 
    def default_prompt(t)
      if t[:until].nil? # no idea when do we end
        msg = "We're doing stuff on our servers and will be back shortly.<br/><br/>"
        msg << "<small>started: #{t[:since]}</small>"
      else
        msg = "We'll be back online on <b>#{t[:until]}</b>"
      end
      "<center><br/><br/><h1>Maintenance.</h1><p>#{msg}</p></center>"
    end
  end
end

Usage

By default middleware will look into current working directory for file with filename “.maintenance”.
Request will be passed next if no file found.

Simpliest example:

use Rack::Maintenance

Set custom directory:

use Rack::Maintenance, '/path/to/dir'

Customize default maintenance prompt:

use Rack::Maintenance do
  "We're doing some stuff. Come back later."
end

If you want to use timestamps in your prompt just put a valid time string
into your .maintenance file:

File.open('.maintenance', 'w') { |f| f.write(Time.now + 3600) }

and then use it in your prompt:

use Rack::Maintenance do |t|
  "We're doing some stuff since #{t[:since} will be done at #{t[:until]}."
end

Results

Here is a small module to bounce off all non-desktop clients (mobile devices, crawlers):

module Rack
  class Restrict
    REGEX_MOBILE = /(blackberry|motorokr|motorola|sony|windows ce|240x320|176x220|palm|mobile|iphone|ipod|symbian|nokia|samsung|midp)/i
    REGEX_BOTS = /(google|yahoo|baidu|bot|webalta|ia_archiver)/
 
    def initialize(app)
      @app = app
    end
 
    def call(env)
      str = env['HTTP_USER_AGENT']
      restrict = !str.match(REGEX_MOBILE).nil? || !str.match(REGEX_BOTS).nil?
      restrict ? forbidden! : @app.call(env)
    end
 
    def forbidden!
      [403, { 'Content-Type' => 'text/html', 'Content-Length' => '0' }, []]
    end
  end
end

Usage: (sample config.ru):

require 'rack'
require 'rack/contrib'
require 'app.rb'
 
use Rack::Restrict
run Sinatra::Application