WebDAV client in ruby 1

Posted by Dan Sosedoff on May 02, 2009

Here is a simple example how to make native WebDAV client with Ruby sockets. No additional gems or extensions needed – just all basic classes.

class WebDAV
	attr_reader :host, :port, :protocol, :chunk_size
	@socket = nil
 
	def initialize(host,port=80,protocol='HTTP/1.1',chunk=8096)
		@host = host.to_s
		@port = port.to_i
		@protocol = protocol
		@chunk_size = chunk.to_i
	end
 
	def build_header(method, path, content_length=nil)
		header = "#{method} #{path} #{@protocol} \r\n"
		header += "Content-Length: #{content_length}\r\n" if !content_length.nil?
		header += "Host: #{@host}\r\n"
		header += "Connection: close\r\n\r\n"
		return header
	end
 
	def request(method, path)
		open
		header = build_header(method, path)
		if @socket.write(header) == header.length then
			return @socket.gets.split[1]
		end
	end
 
	def delete(path)
		request('DELETE', path)
	end
 
	def head(path)
		request('HEAD', path)
	end
 
	def mkcol(path)
		request('MKCOL', path)
	end
 
	def put(path, localfile, auto_head=true)
		if !File.exists?(localfile) || !File.readable?(localfile)
			raise "File not exists or not accessible for reading!"
		end
 
		open
 
		datalen = File.size(localfile)
		header = build_header('PUT', path, datalen)
 
		begin
			if @socket.write(header) == header.length then
				written = 0
				File.open(localfile,'r') do |f| 
					until f.eof? do
						written += @socket.write(f.read(@chunk_size))
					end
				end
 
				if written == datalen
					close
					if !auto_head
						return true
					else
						return head(path)
					end
				end
			end
		rescue Exception => e
			puts e
			return false
		end
	end
 
	def open
		begin 
			@socket = TCPSocket.open(@host,@port)
			return true
		rescue Exception => e
			puts e
			return false
		end
	end
 
	def close
		begin
			return @socket.close
		rescue 
			return false
		end
	end
end

This class supports only basic http/dav methods (PUT, DELETE, MKCOL, HEAD) and can be extended very easily and designed to work with all files, reading them by small chunks (default is 8096 bytes).
Im using this class sometimes with nginx.

Deps:

require 'socket'
require 'digest'

Usage:

# create connection
conn = WebDAV.new('your.host.com')
 
# upload file (without autocheck), return true/false value
result = conn.put('/test.mp3','/home/.../..../..../file.mp3', false)
 
# upload file with autocheck, returns http response code (201, 404, ... ) so you`ll know what exactly happened
result = conn.put('/test2.mp3','/home/.../file.mp3')

Also, here is a wrapper class to produce MD5, SHA1 file hashes that supports big files.

class FileHash 
	def self.md5(path)
		d = Digest::MD5.new
		File.open(path,'r') do |f| 
			d.update(f.read(8192)) until f.eof?
		end
		return d.hexdigest
	end
 
	def self.sha1(path)
		d = Digest::SHA1.new
		File.open(path,'r') do |f| 
			d.update(f.read(8192)) until f.eof?
		end
		return d.hexdigest
	end
end

Usage:

FileHash.md5('/path/to/file')
FileHash.sha1('/path/to/file')

This webdav class not pretending to be stable in production environment, but can be useful for some “one-time” tasks with less code.

inet_ntoa and inet_aton in Ruby

Posted by Dan Sosedoff on April 16, 2009

Here is two small functions to convert IP addresses from string representation to integer and vice versa.

def inet_aton(ip)
    ip.split(/\./).map{|c| c.to_i}.pack("C*").unpack("N").first
end
 
def inet_ntoa(n)
    [n].pack("N").unpack("C*").join "."
end

Simple file uploader to Amazon S3 Service 1

Posted by Dan Sosedoff on March 22, 2009

For a long time i was thinking that Amazon`s Simple Storage Service (S3) is very complicated thing. But, it was before i tried it. Couple days ago, i got account to S3 and started exploring API`s and architecture. Now i see how stupid i was :) It`s really easy to handle all operations with files and buckets. Pricing also comfortable.

Welcome to cloud computing! :) I started using it with Ruby. Regular gem and docs can be found at http://amazon.rubyforge.org/

So, the first useful tool i decided to created – simple uploader of local files to amazons server.
First, we need to create bucket and make it public:

Bucket.create('NAME_HERE',:access => :public_read)

Here`s the client ruby script:

#!/usr/bin/ruby
 
require 'rubygems'
require 'aws/s3'
 
include AWS::S3
 
$s3_bucket = "BUCKET_NAME"
$s3_key = "API_KEY"
$s3_secret = "API_SECRET"
 
def s3_store(localfile)
	if File.exists?(localfile) && File.readable?(localfile)
		puts "Uploading file [#{localfile}]. Size: #{File.size(localfile)} bytes."
		name = File.basename(localfile)
		Base.establish_connection!(:access_key_id => $s3_key, :secret_access_key => $s3_secret)
		S3Object.store(name, open(localfile), $s3_bucket, :access => :public_read)
		puts "Download link: http://s3.amazonaws.com/#{$s3_bucket}/#{name}"
	else
		puts "File not exists or not accessible. Please check file and try again!"
	end
end
 
path = ARGV[0]
if !path
	"Please specify the file to upload."
else
	s3_store(path)
end

Download script: http://files.sosedoff.com/036cfedd/

BTW, I found cool firefox add-on to manage S3 objects/files. It`s pretty easy.
Link to extension – http://www.s3fox.net
Screenshot:

Scaling images with RMagick

Posted by Dan Sosedoff on March 14, 2009

Simple class that providing scaling (rectangle and thumbnails) for images using RMagick and Ruby.
Code:

class ImageScale
    def change_geometry(sz,value)
        w = sz[0] ; h = sz[1]
            if w > h
                sz[0] = value
                sz[1] = ((value * h) / w).floor
            else
                sz[1] = value
                sz[0] = ((value * w) / h).floor
            end
            return sz
    end
 
    def make_rect(file_in,file_out, width, quality=85, sharp=false) 
        if FileTest.exists?(file_in)
            begin
                img = Magick::Image.read(file_in).first
                img.crop_resized!(width,width, Magick::CenterGravity)
                img = img.sharpen(0.5, 0.5) if sharp
                img.write(file_out) { self.quality = quality }
                return true if FileTest.exists?(file_out)
            rescue Magick::ImageMagickError
                return false
            end
        end
        return false
    end
 
    def make_thumb(file_in,file_out, width_to, quality=85, sharp=false) 
        if FileTest.exists?(file_in)
            begin
                img = Magick::Image.read(file_in).first
                info = [img.columns,img.rows]
                sz = change_geometry(info, width_to)
                img = img.resize(sz[0],sz[1])
                img = img.sharpen(0.5, 0.5) if sharp
                img.write(file_out) { self.quality = quality }
                return true if FileTest.exists?(file_out)
            rescue Magick::ImageMagickError
                return false
            end
        end
        return false
    end
end

Ok, let`s see how this class working. For example, we have source image:
Source Image

Function ImageScale.make_rect(src,dest,64) will produce such image:
Rectangle Image

Function ImageScale.make_thumb(src,dest,200) will produce thumbnail:
Thumbnail Image

There is optional parameter sharp to use sharping. Optional parameter quality is set to 85% compression value.

Fetching album covers from Last.Fm API 3

Posted by Dan Sosedoff on February 15, 2009

As previous post was about fetching covers media from Amazon Web Services, this post will be about fetching covers from popular music site – Last.fm. API documentation page

#!/usr/bin/ruby
 
require 'rubygems'
require 'net/http'
require 'cgi'
require 'xmlsimple'
 
# key from API documentation
$lastfm_key = "b25b959554ed76058ac220b7b2e0a026" 
$lastfm_host = "ws.audioscrobbler.com"
 
def fetch_cover(artist, album)
	artist = CGI.escape(artist)
	album = CGI.escape(album)
 
	path = "/2.0/?method=album.getinfo&api_key=#{$lastfm_key}&artist=#{artist}&album=#{album}"
	data = Net::HTTP.get($lastfm_host, path)
	xml = XmlSimple.xml_in(data)
	if xml['status'] == 'ok' then
		album = xml['album'][0]
 
		cover = {
			:small => album['image'][1]['content'],
			:medium => album['image'][2]['content'],
			:big => album['image'][3]['content']
		}
 
		return cover
	end
 
	return nil
end
 
puts fetch_cover('Nickelback', 'Dark Horse').inspect

Download ruby script

Fetching album covers from Amazon Web Service 1

Posted by Dan Sosedoff on February 15, 2009

On my small project i was looking for web service to get media covers from. I found that i can use Amazon Web Services API. The documentation for this ECommerce Service is pretty old, but it still works.
More detailed information about API you can find here

#!/usr/bin/ruby
 
require 'rubygems'
require 'net/http'
require 'cgi'
require 'xmlsimple'
 
$amazon_key = "12DR2PGAQT303YTEWP02" # NOT MY KEY (FOUND ON INTERNET)
$amazon_host = "webservices.amazon.com"
 
def fetch_cover(artist, album)
	artist = CGI.escape(artist)
	album = CGI.escape(album)
 
	path = "/onca/xml?Service=AWSECommerceService&AWSAccessKeyId=#{$amazon_key}&Operation=ItemSearch&SearchIndex=Music&Artist=#{artist}&ResponseGroup=Images&Keywords=#{album}"
	data = Net::HTTP.get($amazon_host, path)
	xml = XmlSimple.xml_in(data)
	if xml['Items'][0]['TotalResults'].to_s.to_i then
		cover = {
			:small => xml['Items'][0]['Item'][0]['SmallImage'][0]['URL'],
			:medium => xml['Items'][0]['Item'][0]['MediumImage'][0]['URL'],
			:big => xml['Items'][0]['Item'][0]['LargeImage'][0]['URL']
		}
		return cover
	end
	return nil
end

So, after execution of this function you will get array with 3 different images (small, medium, big).
I use XML-Simple gem for ruby. Can be installed this way

sudo gem install xml-simple

That`s it. Download script

Writing very simple daemon in Ruby 1

Posted by Dan Sosedoff on January 24, 2009

Ruby is very powerful language, not only for web development as many can think for the first time. It is also providing all necessary resources to build system utilities and daemons. So, this post exactly about it.
Long time ago i was looking for some tool to write simple daemon in a short term. I didn`t choose C just because i was able to develop program only on special computer (i mean “production-like” environment) and the problem wasnt critical to resources. In other words – this daemon was working only few days to complete the task. Ok, lets see what we`ve got:

#!/usr/bin/ruby
# ---------------------------------------------------------------------
# MODULES
# ---------------------------------------------------------------------
require 'daemonize'
include Daemonize
include Process
# ---------------------------------------------------------------------
# CONFIGURATION
# ---------------------------------------------------------------------
$daemon = {
  :name => "Test Daemon",                  # daemon name
  :abbr => "testd",                        # daemon abbreviation
  :author => "(c) 2008 author",            # daemon author
  :version => "0.1",                       # actual version
  :file_log => "/var/log/testdaemon.log",  # log path
  :file_pid => "/var/run/testdaemon.pid",  # process id path
  :delay_sleep => 1,                       # seconds
  :user => 'tux',                          # working data user
  :grp => 'tux',                           # working data group
  :background => false,                    # background mode
  :work => true                            # daemon work flag
}
 
$daemon_log = nil
$daemon_pid = nil
 
# ---------------------------------------------------------------------
 
def daemon_log(str)
  puts "[#{Time.now.strftime("%m/%d/%Y-%H:%M:%S")}] #{str}"
end
 
def daemon_terminate
  $daemon[:work] = false
end
 
def daemon_stop
  daemon_log("Stopping working process...")
  $daemon_pid.close
  File.delete($daemon[:file_pid])
end
 
def daemon_start
  if File.exist?($daemon[:file_pid]) then
    daemon_log("Process already running. If it`s not - remove the pid file")
    exit
  end
 
  daemon_log("Starting process...")
  daemonize if $daemon[:background]
 
  begin
    $daemon_pid = File.new($daemon[:file_pid],"w")
  rescue Errno::EACCES
    daemon_log("Cannot create PID file. Check the permissions and try again!")
    $daemon_pid = nil
    exit
  end  
 
  daemon_work
end
 
def daemon_work
  if ($daemon_pid) then
    $daemon_pid.sync = true
    $daemon_pid.puts(Process.pid.to_s)
 
    begin
      while $daemon[:work] do
        daemon_log("Daemon working")
        daemon_handle_signals
        sleep($daemon[:delay_sleep])
      end
    rescue Exception => e
      daemon_log("Error: #{e.message}")
    end
 
    daemon_stop
  end
end
 
def daemon_handle_signals
  # termination signal
  Signal.trap("TERM") do
    daemon_log("TERM signal received.")
    daemon_terminate
  end  
 
  # kill signal
  Signal.trap("KILL") do
    daemon_log("KILL signal received.")
    daemon_terminate
  end
 
  # keyboard interruption
  Signal.trap("INT") do
    daemon_log("SIGINT signal received.")
    daemon_terminate
  end
 
  Signal.trap("TSTP") do
    daemon_log("SIGTSTP signal received.")
  end
end
 
def daemon_show_version
  puts "#{$daemon[:name]} v#{$daemon[:version]} #{$daemon[:author]}"
end
 
def daemon_show_usage
  daemon_show_version
  puts "Usage:"
  puts "    -b, --background        work in background mode"
  puts "    -v, --version           view version of daemon"
  puts "    -h, --help              view this help"
end
 
def daemon_parse_opts
  return true if ARGV.length == 0
 
  case ARGV[0]
    when '-b', '--background'
      $daemon[:background] = true;
      return true
 
    when '-v', '--version'
      daemon_show_version
 
    when '-h', '--help'
      daemon_show_usage
    else
      puts "Invalid argument: #{ARGV[0]}" if !ARGV[0].nil?
      daemon_show_usage
  end
 
  return false
end
 
def daemon_main
  daemon_start if daemon_parse_opts
end
 
daemon_main

This is just a basic structure of daemon, it supports background mode. I removed all unimportant information and left only main program cycles. As you can see, the entry point of whole program – daemon_main procedure. It parsing the command line parameters. In this example there is no required parameters, so daemon will run in basic mode (not background). To enable background you should specify the -d (or –background) option. Also, very important – this example needs to be executed under root or other user that have access to /var/run. Other way, PID file path can be changed to whatever you want (file_pid key). All daemon configuration variables stored in array $daemon. Also, it supports system signals handling, like SIGTERM or SIGKILL (all ruby signal constants u can explore here).

I think that`s it. Not a lot, but very simple. Download script: http://files.sosedoff.com/b57415aa/

Simple MySQL backup script

Posted by Dan Sosedoff on January 15, 2009

There is a small useful ruby script to backup your MySQL databases in small projects, where speed of backup not so important.
Source:

#!/usr/bin/ruby
# MySQL Backup Utility
# Usage: ./mysql_backup.rb or ruby mysql_backup.rb
 
$backup_archive = true # gzip files after processing
$backup_dir = "/home/storage/backup/" # output directory
$backup_template = "project-%s-%s.sql" # text-%dbname-%timestamp.sql
$backup_cmd = "mysqldump -u local_backup --add-drop-table --databases %s > %s"
$backup_dblist = [ # list of databases to backup
  'main',
  'users',
  'admin',
  'cards'
] 
 
def backup_database(database)
  time = Time.now()
  time_str = sprintf("%02i-%02i-%04i-%02i%02i%02i",time.day, time.month, time.year, time.hour, time.min, time.sec)
  filename = sprintf($backup_template,database,time_str)
  filename = "#{$backup_dir}#{filename}"
 
  cmd = sprintf($backup_cmd,database,filename)
  if system(cmd) then
    system("gzip --best #{filename}") if $backup_archive
  end
end
 
$backup_dblist.each do |db|
  puts "Processing database... #{db}"
  backup_database(db)
end

Paste: http://pastie.org/341839

Next Entries »