If you’re rolling deployments with capistrano, it will insert the REVISION file under the app’s root dir with git sha or svn revision or whatever tag based on SCM of your choice.

Here is the simple rack middleware that injects that revision as a ‘X-REVISION’ header in responses.

module Rack
  class Revision
    @@revision = nil
    File = ::File
 
    def initialize(app, &block)
      @app   = app
      @block = block
      @file  = File.join(Dir.pwd, 'REVISION')
    end
 
    def call(env)
      status, headers, body = @app.call(env)
      headers['X-Revision'] = revision
      [status, headers, body]
    end
 
    protected
 
    def revision
      @@revision ||= read_revision
    end
 
    def read_revision
      File.exists?(@file) ? File.read(@file).strip : 'UNDEFINED'
    end
  end
end

For instance, just put that file as lib/rack/revision.rb and change your config.ru file:

require ::File.expand_path('../config/environment',  __FILE__)
use Rack::Revision
run YOUR_APP::Application

You’ll need to restart the app to apply the changes.

To test if it works just run:

curl -i -X HEAD "http://YOUR_WEBSITE/"

Sample output would be:

ETag: "8d6228d86642025c31e3d54e9a67b14b"
Cache-Control: max-age=0, private, must-revalidate
X-Runtime: 0.001491
X-Rack-Cache: miss
X-Revision: f8c51630843898e88848261dd5ebac3fdebc5e48

I use capistrano for all my apps and find it very useful for quick and handy deployments. After switch to unicorn i’ve been copying-and-pasting the same code over and over again, so i decided to move it to a separate gem.

Check out code on github – capistrano-unicorn.

Usage

Assuming that you already have your capistrano script ready, add gem to the Gemfile:

group :development do
  gem 'capistrano-unicorn'
end

Then, add requirement to the deploy.rb file (after all hooks):

require 'capistrano-unicorn'

Configuration

All unicorn configs should be placed under APP_ROOT/config/unicorn/ENVIRONMENT.rb
So, for each environment there should be a separate file.

Test if that worked:

cap unicorn:start
cap unicorn:stop
cap unicorn:reload

For more information check out github repository – capistrano-unicorn.

It is still under development, but basic functionality works fine.

Capistrano versions: >= 2.5

Configuration

First, you need to define application environment:

# Define server-side rails environment
set :rails_env, "production"
 
# Primary deployment location
set :deploy_to, "/home/#{user}/#{application}"
 
# Place where all backups will be dumped
set :backup_to, "/home/#{user}/#{application}/backups"

Also, add this function. It allows capistrano to check if remote file exists:

def remote_file_exists?(full_path)
  'true' ==  capture("if [ -e #{full_path} ]; then echo 'true'; fi").strip
end

Now, we need to add a backup task:

namespace :utils do
  desc 'Backup database before deploy'
  task :backup, :roles => :db, :only => {:primary => true} do
    run "mkdir -p #{backup_to}" # Create a backup folder unless exists
 
    # Primary backup filename
    filename = "#{backup_to}/#{application}_predeploy_#{Time.now.strftime("%m%d%Y%H%I%S")}.sql.gz"
 
    # Check if we've got database config
    if remote_file_exists?("#{deploy_to}/current/config/database.yml")    
      text = capture("cat #{deploy_to}/current/config/database.yml")
      config = YAML::load(text)[rails_env]
 
      on_rollback { run "rm #{filename}" }
      run "mysqldump -u #{config['username']} -p #{config['database']} | gzip --best > #{filename}" do |ch, stream, out|
        ch.send_data "#{config['password']}\n" if out =~ /^Enter password:/
      end
    else
      logger.debug("[Backup] No configuration file was found.")
    end
  end
end

And finally, add the capistrano before hook:

before "deploy", "utils:backup"

Testing

To test if the task works, run:

cap utils:backup

On the server side you should see a backup file:

/home/USER/APP/backups/APP_predeploy_MMDDYYHHMMSS.sql.gz.

Stack:

• Debian / Ubuntu Server
• Postix
• Ruby 1.9
• Rails 3.0

Overview

You have an application where users get email notifications. And you want to allow them to reply directly to the email.
In order to do so, each email should have an unique (depends on situation) reply-to address. Usually its something like that:

P946d272cf7da4dd6b0cb613605bced65@yourdomain.com

This means that the mailserver you use should support dynamic/virtual email addresses and forwarding.

Postfix Configuration

First, you’ll need to install postfix (in not installed):

apt-get install postfix

Configuration should look like this:

# See /usr/share/postfix/main.cf.dist for a commented, more complete version

default_privs = apps

# Debian specific:  Specifying a file name will cause the first
# line of that file to be used as the name.  The Debian default
# is /etc/mailname.
#myorigin = ap

smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
biff = no

# appending .domain is the MUA's job.
append_dot_mydomain = no

# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h

readme_directory = no

# TLS parameters
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_use_tls=no
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

# See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for
# information on enabling SSL in the smtp client.

myhostname = YOUR_APP_HOSTNAME
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydomain = YOUR_APP_DOMAIN
mydestination = YOUR_APP_DOMAIN, localhost
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
relay_domain = localhost
recipient_canonical_maps = regexp:/etc/postfix/recipient_canonical

Last option recipient_canonical_maps allows you to define a dynamic email addresses and forward them to the specific system mailbox for processing.

Create a file /etc/postfix/recipient_canonical:

/^P[0-9abcdef]{1,}(M[0-9]{1,})?/ apps

This will add a virtual recipient addresses and forward messages to user apps.

NOTE: Regular expressions should be in POSIX format. For test you can use regextester.com

Email Aliases

After you added support for virtual addresses all mail will be delivered to the system user mailbox (apps). But, we need to drive all that traffic into our app. In order to do so we will have to setup mail piping directly into your application script.

Edit /etc/aliases:

apps: "| /home/apps/APP_NAME/current/script/email_receiver_script"

And rebuild the aliases db by running:

newaliases

Do not forget to restart postfix:

/etc/init.d/postfix restart

You can test out the email delivery. For errors check /var/log/mail.info

Mail Receiver Script

Since all mail will be forwarded directly to our mail receiver script via piping there are few things to consider:

• Email receiver should consume as less memory as possible.
• Email receiver should not load the whole application (because of item above).
• Email receiver should only validate and preprocess incoming messages and leave actual processing to another subsystem via queue.

Configuration

There are few ruby libraries that are well suited for this case:

• mail – Email processing, ruby 1.9.2 compatible (comparing to tmail which is not)
• redis – Simple key-value in-memory database.
• resque – Redis-backed library for creating background jobs, placing those jobs on multiple queues, and processing them.

Install gems:

gem install mail redis resque

Here is an example email receiver script:

#!/usr/bin/env ruby 

require 'rubygems'
require 'mail'
require 'redis'
require 'resque' 

class EmailReply
  @queue = :email_replies 

  def initialize(content)
    mail    = Mail.read_from_string(content)
    from    = mail.from.first
    to      = mail.to.first 

    if mail.multipart?
      part = mail.parts.select { |p| p.content_type =~ /text\/plain/ }.first rescue nil
      unless part.nil?
        message = part.body.decoded
      end
    else
      message = part.body.decoded
    end 

    unless message.nil?
      Resque.enqueue(EmailReply, from, to, message)
    end
  end
end 

EmailReply.new($stdin.read)

This script receives the mail message then tries to extract the plaintext body. If the email message is valid it adds it to the queue for future processing.

Mail Queue processing

After we put emails into the queue we’ll need to create a worker.

If you need to extract a reply from the body, use (mail_extract)[https://github.com/sosedoff/mail_extract]:

gem install mail_extract

Simple worker (resque job worker), extracted from one of the projects. (RAILS_ROOT/lib/email_reply.rb):

class InvalidReplyUUID    < StandardError ; end
class InvalidReplyUser    < StandardError ; end
class InvalidReplyProject < StandardError ; end
class InvalidReplyMessage < StandardError ; end 

class EmailReply
  @queue = :email_replies 

  def self.parse_email_uuid(str)
    if str =~ /^P[0-9abcdef]+(M[\d]+)?@/i
      parts = str.scan(/^P([0-9abcdef]+)(M([\d]+))?/).flatten
      project_uuid = parts.first
      message_id = parts.size == 3 ? parts.last : nil 

      result = {:project_uuid => project_uuid}
      result[:message_id] = message_id unless message_id.nil?
      result
    else
      raise InvalidReplyUUID, "Invalid UUID: #{str}"
    end
  end 

  def self.perform(from, to, body)
    user = User.find_by_email(from)
    if user.nil?
      raise InvalidReplyUser, "User with email = #{from} is not a member of the app."
    end 

    info = parse_email_uuid(to) 

    project = Project.find_by_uuid(info[:project_uuid])
    if project.nil?
      raise InvalidReplyProject, "Project with UUID = #{info[:project_uuid]} was not found."
    end 

    if info.key?(:message_id)
      message = project.messages.find_by_id(info[:message_id])
      if message.nil?
        raise InvalidReplyMessage, "Message with ID = #{info[:message_id]} was not found on project '#{project.name}'"
      end
    end 

    params = {
      :project  => project,
      :body     => MailExtract.new(body).body,
      :markup   => 'plain',
      :sent_via => 'email'
    }
    params[:message] = message unless message.nil? 

    message = user.messages.new(params)
    unless message.save
      raise RuntimeError, "Unable to save message. Errors: #{message.errors.inspect}"
    end
  end
end

NOTE: Its important that both mail receiver and worker are using the same queue.

Create a resque.rake in RAILS_ROOT/lib/tasks:

require 'resque/tasks'
task "resque:setup" => :environment

And fire it up:

rake resque:work QUEUE=email_replies

Source: https://github.com/ddollar/foreman

Installation

gem install foreman

Configuration

Lets assume you have a resque worker, which you execute with rake:

bundle exec rake resque:work QUEUE=* RAILS_ENV=production

Now, you will need to create a Procfile inside your rails root folder (Procfile):

worker: bundle exec rake resque:work QUEUE=* RAILS_ENV=production

Start your workers with foreman:

foreman start

Foreman will start all of the processes associated with your app and display stdout and stderr of each process. Processes are color-coded by type to make them easy to read. $PORT will be automatically assigned by foreman and made available to each underlying process.

Export to system

While foreman is great for running your application in development, there are some great existing tools for managing processes in production. Foreman can export to either upstart or standard unix init.

Exporting to upstart:

$ foreman export upstart /etc/init

[foreman export] writing: /etc/init/testapp.conf
[foreman export] writing: /etc/init/testapp-web.conf
[foreman export] writing: /etc/init/testapp-web-1.conf

[foreman export] writing: /etc/init/testapp-worker.conf
[foreman export] writing: /etc/init/testapp-worker-1.conf
[foreman export] writing: /etc/init/testapp-clock.conf

[foreman export] writing: /etc/init/testapp-clock-1.conf

Capistrano integration

Assuming you got your capistrano recipe working, here is a set of foreman tasks:

# Foreman tasks

namespace :foreman do
  desc 'Export the Procfile to Ubuntu upstart scripts'
  task :export, :roles => :app do

    run "cd #{release_path} && sudo bundle exec foreman export upstart /etc/init -a #{application} -u #{user} -l #{release_path}/log/foreman"

  end

  desc "Start the application services"
  task :start, :roles => :app do

    sudo "start #{application}"
  end

  desc "Stop the application services"

  task :stop, :roles => :app do
    sudo "stop #{application}"

  end

  desc "Restart the application services"
  task :restart, :roles => :app do

    run "sudo start #{application} || sudo restart #{application}"
  end
end

Just add deploy hooks and its ready to go:

after "deploy:update", "foreman:export"    # Export foreman scripts
after "deploy:update", "foreman:restart"   # Restart application scripts

Documentation

• Github repo – https://github.com/ddollar/foreman
• Blog post – http://blog.daviddollar.org/2011/05/06/introducing-foreman.html

Here is a Rack::Maintenance module that provides such functionality:

module Rack
  class Maintenance
    File = ::File
 
    def initialize(app, dir=nil, &block)
      @app = app
      @block = block
      @dir = dir || Dir.pwd
      @file = File.join(File.expand_path(@dir), '.maintenance')
    end
 
    def call(env)
      if File.exists?(@file)
        body = @block.nil? ? default_prompt(time_info) : @block.call(time_info)
        res = Response.new
        res.write(body)
        res.finish
      else
        @app.call(env)
      end
    end
 
    private
 
    def time_info
      t_since, t_until = [nil, nil]
      File.open(@file) do |f|
        t_since = f.mtime
        t_until = Time.parse(f.gets) rescue nil
        t_until = nil if t_until.kind_of?(Time) && t_until < Time.now
      end
      {:since => t_since, :until => t_until}
    end
 
    def default_prompt(t)
      if t[:until].nil? # no idea when do we end
        msg = "We're doing stuff on our servers and will be back shortly.<br/><br/>"
        msg << "<small>started: #{t[:since]}</small>"
      else
        msg = "We'll be back online on <b>#{t[:until]}</b>"
      end
      "<center><br/><br/><h1>Maintenance.</h1><p>#{msg}</p></center>"
    end
  end
end

Usage

By default middleware will look into current working directory for file with filename “.maintenance”.
Request will be passed next if no file found.

Simpliest example:

use Rack::Maintenance

Set custom directory:

use Rack::Maintenance, '/path/to/dir'

Customize default maintenance prompt:

use Rack::Maintenance do
  "We're doing some stuff. Come back later."
end

If you want to use timestamps in your prompt just put a valid time string
into your .maintenance file:

File.open('.maintenance', 'w') { |f| f.write(Time.now + 3600) }

and then use it in your prompt:

use Rack::Maintenance do |t|
  "We're doing some stuff since #{t[:since} will be done at #{t[:until]}."
end

Results

Here is a small module to bounce off all non-desktop clients (mobile devices, crawlers):

module Rack
  class Restrict
    REGEX_MOBILE = /(blackberry|motorokr|motorola|sony|windows ce|240x320|176x220|palm|mobile|iphone|ipod|symbian|nokia|samsung|midp)/i
    REGEX_BOTS = /(google|yahoo|baidu|bot|webalta|ia_archiver)/
 
    def initialize(app)
      @app = app
    end
 
    def call(env)
      str = env['HTTP_USER_AGENT']
      restrict = !str.match(REGEX_MOBILE).nil? || !str.match(REGEX_BOTS).nil?
      restrict ? forbidden! : @app.call(env)
    end
 
    def forbidden!
      [403, { 'Content-Type' => 'text/html', 'Content-Length' => '0' }, []]
    end
  end
end

Usage: (sample config.ru):

require 'rack'
require 'rack/contrib'
require 'app.rb'
 
use Rack::Restrict
run Sinatra::Application

AppConfig is a library to manage your (web) application dynamic settings with flexible access and configuration strategy. Primary datasource for AppConfig is an ActiveRecord model.

Installation

git clone git://github.com/sosedoff/app-config.git
cd app-config
gem build
gem install app-config-x.y.z.gem

Data Formats

You can use following formats:

• String
• Boolean
• Array
• Hash

String format is a default format. Everything is a string by default.

Boolean format is just a flag, values ‘true’, ‘on’, ‘yes’, ‘y’, ‘1’ are equal to True. Everything else is False.

Array format is a multiline text which is transformed into array. Each evelemnt will be trimmed. Empty strings are ignored.

Hash format is special key-value string, “foo: bar, user: username”, which is transformed into Hash instance. Only format “keyname: value, keyname2: value2″ is supported. No nested hashes allowed.

Usage

AppConfig is designed to work with ActiveRecord model. Only ActiveRecord >= 3.0.0 is supported.
By default model “Setting” will be used as a data source.

Here is default structure:

ActiveRecord::Schema.define do
  create_table :settings do |t|
    t.string :keyname
    t.string :value
    t.string :value_format
  end
end

Now, configure:

AppConfig.configure

If your settings model has a different schema, you can redefine columns:

AppConfig.configure(
  :model  => Setting,           # define your model as a source
  :key    => 'KEYNAME_FIELD',   # field that contains name
  :format => 'FORMAT_FIELD',    # field that contains key format
  :value  => 'VALUE_FIELD',     #field that contains value data
)

Load all settings somewhere in your application. In Rails it should be initializer file.

AppConfig.load

AppConfig gives you 2 ways to access variables:

AppConfig.my_setting      # method-like
AppConfig[:my_setting]    # hash-like by symbol key
AppConfig['my_setting']   # hash-like by string key

You can define settings items manually. NOTE: THESE KEYS WILL BE REMOVED ON RELOAD/LOAD.

AppConfig.set('KEYNAME, 'VALUE', 'FORMAT')

Everytime you change your settings on the fly, use reload:

AppConfig.reload

Cleanup everything:

AppConfig.flush

Github source: https://github.com/sosedoff/app-config

The documentation is not the perfect, but enough to understand the concepts, so i’ll just jump straight to the matter.

Subject

Lets assume that all our templates are stored in database and we dont store anything in filesystem. And we need to use small parts as snippets to embed into actual template.

Implementation

The model where we store all snippets with be named Snippet. Fields:

• ID – Primary key
• Name – Actual name of the snippet used in tempates
• Body – Content of the snippet
• Created/Updated timestamps – DateTime stamps for internal use

And the active record model:

class Snippet < ActiveRecord::Base
  NAME_FORMAT = /^[a-z\d\_\-]{2,64}$/i    # Define your own name format
 
  validates_presence_of   :name, :body
  validates_length_of     :name, :within => 2..64
  validates_format_of     :name, :with => NAME_FORMAT
  validates_uniqueness_of :name, :case_sensitive => false
end

Now, lets define the custom liquid tag “snippet”:

class SnippetTag < Liquid::Tag
  def initialize(tag_name, snippet_name, tokens)
    super
    @name = snippet_name
  end
 
  def render(context)
    # You might want to define caching mechanism here
    snippet = Snippet.find_by_name(@name)
    unless snippet.nil?
      Liquid::Template.parse(snippet.body).render
    else
      "Error: Snippet #{@name} was not found!"
    end
  end
end

Now, register the tag:

Liquid::Template.register_tag('snippet', SnippetTag)

And use it within your template:

This is a custom tag that renders a widget: {% snippet sample_snippet %}

If the requested snippet was not found or does not exists renderer will replace the tag with error message, which might be adjusted for your personal needs or hidden. Also, you can use nested snippets, but in this case you have to think about right snippet template caching approach.

Using Liquid drops

A drop in liquid is a class which allows you to use predefined data collection methods in the template. If you would like to make data available to the web designers which you don’t want loaded unless needed then a drop is a great way to do that.

Lets make a drop for a basic dataset for Post model (blog posts for example):

class PostsDrop < Liquid::Drop
  # fetch all posts
  def all
    Post.all
  end
 
  # fetch N most popular posts
  def popular(num=10)
    Post.all(:order => 'pageviews DESC', :limit => num)
  end
end

Define a template for this collection:

Popular posts:

{% for post in posts.popular %}
Post: {{ post.title }}
{% endfor %}

Load a drop into the template:

template = Liquid::Template.parse(YOUR_TEMPLATE).render('posts' => PostsDrop.new)

With such scheme you can hook up custom snippet that renders a list of objects from your drop.
Pretty cool. Works on Rails 3.

• Firefox LiveHTTP Headers extension
• Charles Web Debugging Proxy Application
• Proxie

Proxie is another project of mine, which i finally extracted from a script written a while ago and published it as a ruby gem.
I used it to track Grooveshark API and it worked out great. Here is the list of features:

• Bind proxy server to any port (default is 8080)
• Define output database. It uses SQLite3 by default, but you can extend it with any other types
• Sinatra-base web interface to browse all your collected data

Installation:

sudo gem install proxie

Start a server:

proxie -d DATABASE_NAME

After data collection just run:

proxie --web

The web interface runs on localhost:4567 by default.

Usage summary:

Usage: proxie [options]
    -i, --info                       Display this information.
    -p, --port PORT                  Listen on port (8080 default)
    -d, --db NAME                    Store results to database
    -w, --web                        Start a Web UI for databases
    -f, --flush                      Delete all local databases

Project on GitHub: http://github.com/sosedoff/proxie