Simple file uploader to Amazon S3 Service 1

Posted by Dan Sosedoff on March 22, 2009

For a long time i was thinking that Amazon`s Simple Storage Service (S3) is very complicated thing. But, it was before i tried it. Couple days ago, i got account to S3 and started exploring API`s and architecture. Now i see how stupid i was :) It`s really easy to handle all operations with files and buckets. Pricing also comfortable.

Welcome to cloud computing! :) I started using it with Ruby. Regular gem and docs can be found at http://amazon.rubyforge.org/

So, the first useful tool i decided to created – simple uploader of local files to amazons server.
First, we need to create bucket and make it public:

Bucket.create('NAME_HERE',:access => :public_read)

Here`s the client ruby script:

#!/usr/bin/ruby
 
require 'rubygems'
require 'aws/s3'
 
include AWS::S3
 
$s3_bucket = "BUCKET_NAME"
$s3_key = "API_KEY"
$s3_secret = "API_SECRET"
 
def s3_store(localfile)
	if File.exists?(localfile) && File.readable?(localfile)
		puts "Uploading file [#{localfile}]. Size: #{File.size(localfile)} bytes."
		name = File.basename(localfile)
		Base.establish_connection!(:access_key_id => $s3_key, :secret_access_key => $s3_secret)
		S3Object.store(name, open(localfile), $s3_bucket, :access => :public_read)
		puts "Download link: http://s3.amazonaws.com/#{$s3_bucket}/#{name}"
	else
		puts "File not exists or not accessible. Please check file and try again!"
	end
end
 
path = ARGV[0]
if !path
	"Please specify the file to upload."
else
	s3_store(path)
end

Download script: http://files.sosedoff.com/036cfedd/

BTW, I found cool firefox add-on to manage S3 objects/files. It`s pretty easy.
Link to extension – http://www.s3fox.net
Screenshot:

Scaling images with ImageMagick and PHP 1

Posted by Dan Sosedoff on March 20, 2009

Here is php version of ruby class that i made a long time ago. The same functionality and results.

<?
class ImageScale {
 
	private function changeGeometry($sz,$value) {
		if ($sz['width'] > $sz['height']) { // horizontal image
			$newsz['width'] = $value;
			$newsz['height'] = ceil(($newsz['width'] * $sz['height']) / $sz['width']);
		}
		else { // vertical image
			$newsz['height'] = $value;
			$newsz['width'] = ceil(($newsz['height'] * $sz['width']) / $sz['height']);
		}
		return $newsz;	
	}
 
	private function changeGeometry2($sz,$value) {
		$newsz['width'] = $value;
		$newsz['height'] = ceil(($newsz['width'] * $sz['height']) / $sz['width']);
		return $newsz;
	}
 
	/**
	 * 	Make thumbnail of specified maximum side size
	 */
	public function processThumb($source_path, $dest_path, $sidesize, $quality=85, $scale_method=1) {
		if (file_exists($source_path) && is_readable($source_path)) {
			$image = new Imagick($source_path);
 
			$geometry = $image->getImageGeometry(); // ['width', 'height']
			if ($geometry['width'] > $sidesize) {
				if ($scale_method == 1) $geometry = $this->changeGeometry($geometry,$sidesize);
				if ($scale_method == 2) $geometry = $this->changeGeometry2($geometry,$sidesize);
			}
			$image->cropThumbnailImage($geometry['width'],$geometry['height']);
			$image->setCompression(Imagick::COMPRESSION_JPEG);
			$image->setCompressionQuality($quality);
			return $image->writeImage($dest_path);
		}
		return false;
	}	
 
	/**
	 * 	Make square thumbnail
	 */
	public function processRectThumb($source_path, $dest_path, $size=150, $quality=85) {
		if (file_exists($source_path) && is_readable($source_path)) {
			$image = new Imagick($source_path);
 
			if ($image) {
				$image->cropThumbnailImage($size,$size);
				$image->setCompression(Imagick::COMPRESSION_JPEG);
				$image->setCompressionQuality($quality);
				return $image->writeImage($dest_path);
			}
		}
		return false;
	}
}
?>

Download file – http://files.sosedoff.com/1167c8db/

Init.d nginx script for Debian and RHEL systems

Posted by Dan Sosedoff on March 15, 2009

Here is general init.d nginx script that working on Debian and RHEL/CentOS systems. It`s set to default configuration with main path to nginx = /usr/local/nginx

#!/bin/sh
#
# Init file for nginx
#
# chkconfig: 2345 55 25
# description: Nginx web server
#
# processname: nginx
# config: /usr/local/nginx/nginx.conf
# pidfile: /usr/local/nginx/nginx.pid
 
# Description: Startup script for nginx webserver on Debian. Place in /etc/init.d and
# run 'sudo update-rc.d nginx defaults', or use the appropriate command on your
# distro. For CentOS/Redhat run: '/sbin/chkconfig --add nginx'
#
# Author: Ryan Norbauer <ryan.norbauer@gmail.com>
# Modified: Geoffrey Grosenbach http://topfunky.com
# Modified: David Krmpotic http://davidhq.com
# Modified: Vishnu Gopal http://vish.in
# Modified: Dan Sosedov <dan.sosedoff@gmail.com>
 
set -e
 
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DESC="nginx daemon"
NAME=nginx
DAEMON=/usr/local/nginx/sbin/$NAME
CONFIGFILE=/usr/local/nginx/conf/nginx.conf
PIDFILE=/usr/local/nginx/logs/$NAME.pid
SCRIPTNAME=/etc/init.d/$NAME
 
# Gracefully exit if the package has been removed.
test -x $DAEMON || exit 0
 
d_start() {
  $DAEMON -c $CONFIGFILE || echo -en "\n already running"
}
 
d_stop() {
  kill -QUIT `cat $PIDFILE` || echo -en "\n not running"
}
 
d_reload() {
  kill -HUP `cat $PIDFILE` || echo -en "\n can't reload"
}
 
case "$1" in
  start)
    echo -n "Starting $DESC: $NAME"
    d_start
        echo "."
  ;;
  stop)
    echo -n "Stopping $DESC: $NAME"
    d_stop
        echo "."
  ;;
  reload)
    echo -n "Reloading $DESC configuration..."
    d_reload
        echo "."
  ;;
  restart)
    echo -n "Restarting $DESC: $NAME"
    d_stop
    # One second might not be time enough for a daemon to stop,
    # if this happens, d_start will fail (and dpkg will break if
    # the package is being upgraded). Change the timeout if needed
    # be, or change d_stop to have start-stop-daemon use --retry.
    # Notice that using --retry slows down the shutdown process somewhat.
    sleep 1
    d_start
    echo "."
  ;;
  *)
    echo "Usage: $SCRIPTNAME {start|stop|restart|reload}" >&2
    exit 3
  ;;
esac
 
exit 0

Usage:

$ sudo /etc/init.d/nginx (start|stop|restart|reload)

You also can download it – http://files.sosedoff.com/e570e29f/

Scaling images with RMagick

Posted by Dan Sosedoff on March 14, 2009

Simple class that providing scaling (rectangle and thumbnails) for images using RMagick and Ruby.
Code:

class ImageScale
    def change_geometry(sz,value)
        w = sz[0] ; h = sz[1]
            if w > h
                sz[0] = value
                sz[1] = ((value * h) / w).floor
            else
                sz[1] = value
                sz[0] = ((value * w) / h).floor
            end
            return sz
    end
 
    def make_rect(file_in,file_out, width, quality=85, sharp=false) 
        if FileTest.exists?(file_in)
            begin
                img = Magick::Image.read(file_in).first
                img.crop_resized!(width,width, Magick::CenterGravity)
                img = img.sharpen(0.5, 0.5) if sharp
                img.write(file_out) { self.quality = quality }
                return true if FileTest.exists?(file_out)
            rescue Magick::ImageMagickError
                return false
            end
        end
        return false
    end
 
    def make_thumb(file_in,file_out, width_to, quality=85, sharp=false) 
        if FileTest.exists?(file_in)
            begin
                img = Magick::Image.read(file_in).first
                info = [img.columns,img.rows]
                sz = change_geometry(info, width_to)
                img = img.resize(sz[0],sz[1])
                img = img.sharpen(0.5, 0.5) if sharp
                img.write(file_out) { self.quality = quality }
                return true if FileTest.exists?(file_out)
            rescue Magick::ImageMagickError
                return false
            end
        end
        return false
    end
end

Ok, let`s see how this class working. For example, we have source image:
Source Image

Function ImageScale.make_rect(src,dest,64) will produce such image:
Rectangle Image

Function ImageScale.make_thumb(src,dest,200) will produce thumbnail:
Thumbnail Image

There is optional parameter sharp to use sharping. Optional parameter quality is set to 85% compression value.

Identify www bots and mobile devices

Posted by Dan Sosedoff on March 12, 2009

Need to identify web crawlers and mobile devices with your web-app ? Here is the list of couple regular expressions you can use (most common values):

Mobile Devices:

/(blackberry|motorokr|motorola|sony|windows ce|240x320|176x220|palm|mobile|iphone|ipod|symbian|nokia|samsung|midp)/i

Web Spyders:

/(google|yahoo|baidu|bot|webalta|archiver|crawler|spyder)/i

Connecting to wifi with specified wep key index

Posted by Dan Sosedoff on March 09, 2009

Since i got trouble while connecting to wireless network with given parameters in linux, i tried a lot of ways to get my internet working.

So, the problem is: Windows Network manager have special option – Key index. The index is transmitted with the encrypted message. The receiver then looks-up the key corresponding to the transmitted index and uses it to decrypt the message. But linux (ubuntu) network managers i`ve tried have no key index field, so there is no way to set it up properly with gui. Connection won`t work if index value set to incorrect value.

After giving up to find any useful gui program i wrote small shell script only for one network (because i have no access to router and no chance to improve some security settings):

#!/bin/bash
 
# settings
interface="ath0" # wireless interface, default to wlan0
essid="NETWORK_NAME_HERE"
key="YOUR_KEY_HERE"
index="4" # can be [1..4]
 
# check permission
if [ "$(id -u)" != "0" ]; then
   echo "Run this script under root" 1>&2
   exit 1
fi
 
# show information
echo "Settings:"
echo "-> Interface: $interface"
echo "-> Wifi ESSID: $essid"
echo "-> Key: $key"
echo "-> Key Index: $index"
 
# perform association
ifconfig $interface down
iwconfig $interface essid $essid
iwconfig $interface key $key [$index]
iwconfig $interface key [$index]
ifconfig $interface up
dhclient $interface

Download shell script – http://files.sosedoff.com/9591756c/

Check root user permissions in bash scripts

Posted by Dan Sosedoff on March 09, 2009

For example, in your bash shell script you`re going to use some root-specific commands like network operations, mounting devices and so on. There are couple easy ways to check if your script is executing under root privileges.

#!/bin/bash
# ...
if [ "$(id -u)" != "0" ]; then
   echo "This script must be run as root" 1>&2
   exit 1
fi
# ...

Another way: use EUID. When user account created a user ID is assigned to each user. Bash shell stores the user ID in $UID variable. Your effective user ID is stored in $EUID variable.

#!/bin/bash
# ...
if [[ $EUID -ne 0 ]]; then
   echo "This script must be run as root" 1>&2
   exit 1
fi
# ...