Snippet: Cleanup your Git repository

Posted by Dan Sosedoff on June 15, 2010

Snippet (found on net) for removing files from repository that are no longer present under your project.

$ git rm $(git ls-files -d)

For best use add it to bash alias file: ~/.bashrc or ~/.bash-aliases (under ubuntu):

alias gitclean='git rm $(git ls-files -d)'

Setting processor affinity for a certain task or process in Linux

Posted by Dan Sosedoff on June 06, 2010

When you are using SMP you might want to override the kernel’s process scheduling and bind a certain process to a specific CPU(s).

What is this?

CPU affinity is nothing but a scheduler property that “bonds” a process to a given set of CPUs on the SMP system. The Linux scheduler will honor the given CPU affinity and the process will not run on any other CPUs. Note that the Linux scheduler also supports natural CPU affinity:

The scheduler attempts to keep processes on the same CPU as long as practical for performance reasons. Therefore, forcing a specific CPU affinity is useful only in certain applications. For example, application such as Oracle (ERP apps) use # of cpus per instance licensed. You can bound Oracle to specific CPU to avoid license problem. This is a really useful on large server having 4 or 8 CPUS

Setting processor affinity for a certain task or process using taskset command

taskset is used to set or retrieve the CPU affinity of a running process given its PID or to launch a new COMMAND with a given CPU affinity. However taskset is not installed by default. You need to install schedutils (Linux scheduler utilities) package.

$ apt-get install shedutils

Under latest version of Debian / Ubuntu Linux taskset is installed by default using util-linux package.

The CPU affinity is represented as a bitmask, with the lowest order bit corresponding to the first logical CPU and the highest order bit corresponding to the last logical CPU. For example:

  • 0×00000001 is processor #0 (1st processor)
  • 0×00000003 is processors #0 and #1
  • 0×00000004 is processors #2 (3rd processor)

To set the processor affinity of process 13545 to processor #0 (1st processor) type following command:

$ taskset 0x00000001 -p 13545

If you find a bitmask hard to use, then you can specify a numerical list of processors instead of a bitmask using -c flag:

$ taskset -c 1 -p 13545
$ taskset -c 3,4 -p 13545

where -p : Operate on an existing PID and not launch a new task (default is to launch a new task)

via http://www.cyberciti.biz/tips/setting-processor-affinity-certain-task-or-process.html

Date separated MySQL backups

Posted by Dan Sosedoff on September 18, 2009

Here is the bash shell script that makes archived dumps of your database server. All databases are separated from each other and stored into date based folders.

#!/bin/bash

MyUSER="root"
MyPASS=""
MyHOST="localhost"
NOW="$(date +"%d-%m-%Y")"
STOREDIR="/home/storage/backup/database/by_dates/$NOW"
DBLIST="$(mysql -u $MyUSER -h $MyHOST -Bse 'show databases')"

[ ! -d $STOREDIR ] && mkdir -p $STOREDIR || :

for db in $DBLIST
do
	FILE="$STOREDIR/$db.gz"
	mysqldump -u $MyUSER -h $MyHOST $db | gzip -9 > $FILE
done

Init.d nginx script for Debian and RHEL systems

Posted by Dan Sosedoff on March 15, 2009

Here is general init.d nginx script that working on Debian and RHEL/CentOS systems. It`s set to default configuration with main path to nginx = /usr/local/nginx

#!/bin/sh
#
# Init file for nginx
#
# chkconfig: 2345 55 25
# description: Nginx web server
#
# processname: nginx
# config: /usr/local/nginx/nginx.conf
# pidfile: /usr/local/nginx/nginx.pid
 
# Description: Startup script for nginx webserver on Debian. Place in /etc/init.d and
# run 'sudo update-rc.d nginx defaults', or use the appropriate command on your
# distro. For CentOS/Redhat run: '/sbin/chkconfig --add nginx'
#
# Author: Ryan Norbauer <ryan.norbauer@gmail.com>
# Modified: Geoffrey Grosenbach http://topfunky.com
# Modified: David Krmpotic http://davidhq.com
# Modified: Vishnu Gopal http://vish.in
# Modified: Dan Sosedov <dan.sosedoff@gmail.com>
 
set -e
 
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DESC="nginx daemon"
NAME=nginx
DAEMON=/usr/local/nginx/sbin/$NAME
CONFIGFILE=/usr/local/nginx/conf/nginx.conf
PIDFILE=/usr/local/nginx/logs/$NAME.pid
SCRIPTNAME=/etc/init.d/$NAME
 
# Gracefully exit if the package has been removed.
test -x $DAEMON || exit 0
 
d_start() {
  $DAEMON -c $CONFIGFILE || echo -en "\n already running"
}
 
d_stop() {
  kill -QUIT `cat $PIDFILE` || echo -en "\n not running"
}
 
d_reload() {
  kill -HUP `cat $PIDFILE` || echo -en "\n can't reload"
}
 
case "$1" in
  start)
    echo -n "Starting $DESC: $NAME"
    d_start
        echo "."
  ;;
  stop)
    echo -n "Stopping $DESC: $NAME"
    d_stop
        echo "."
  ;;
  reload)
    echo -n "Reloading $DESC configuration..."
    d_reload
        echo "."
  ;;
  restart)
    echo -n "Restarting $DESC: $NAME"
    d_stop
    # One second might not be time enough for a daemon to stop,
    # if this happens, d_start will fail (and dpkg will break if
    # the package is being upgraded). Change the timeout if needed
    # be, or change d_stop to have start-stop-daemon use --retry.
    # Notice that using --retry slows down the shutdown process somewhat.
    sleep 1
    d_start
    echo "."
  ;;
  *)
    echo "Usage: $SCRIPTNAME {start|stop|restart|reload}" >&2
    exit 3
  ;;
esac
 
exit 0

Usage:

$ sudo /etc/init.d/nginx (start|stop|restart|reload)

You also can download it – http://files.sosedoff.com/e570e29f/

Connecting to wifi with specified wep key index

Posted by Dan Sosedoff on March 09, 2009

Since i got trouble while connecting to wireless network with given parameters in linux, i tried a lot of ways to get my internet working.

So, the problem is: Windows Network manager have special option – Key index. The index is transmitted with the encrypted message. The receiver then looks-up the key corresponding to the transmitted index and uses it to decrypt the message. But linux (ubuntu) network managers i`ve tried have no key index field, so there is no way to set it up properly with gui. Connection won`t work if index value set to incorrect value.

After giving up to find any useful gui program i wrote small shell script only for one network (because i have no access to router and no chance to improve some security settings):

#!/bin/bash
 
# settings
interface="ath0" # wireless interface, default to wlan0
essid="NETWORK_NAME_HERE"
key="YOUR_KEY_HERE"
index="4" # can be [1..4]
 
# check permission
if [ "$(id -u)" != "0" ]; then
   echo "Run this script under root" 1>&2
   exit 1
fi
 
# show information
echo "Settings:"
echo "-> Interface: $interface"
echo "-> Wifi ESSID: $essid"
echo "-> Key: $key"
echo "-> Key Index: $index"
 
# perform association
ifconfig $interface down
iwconfig $interface essid $essid
iwconfig $interface key $key [$index]
iwconfig $interface key [$index]
ifconfig $interface up
dhclient $interface

Download shell script – http://files.sosedoff.com/9591756c/

Check root user permissions in bash scripts

Posted by Dan Sosedoff on March 09, 2009

For example, in your bash shell script you`re going to use some root-specific commands like network operations, mounting devices and so on. There are couple easy ways to check if your script is executing under root privileges.

#!/bin/bash
# ...
if [ "$(id -u)" != "0" ]; then
   echo "This script must be run as root" 1>&2
   exit 1
fi
# ...

Another way: use EUID. When user account created a user ID is assigned to each user. Bash shell stores the user ID in $UID variable. Your effective user ID is stored in $EUID variable.

#!/bin/bash
# ...
if [[ $EUID -ne 0 ]]; then
   echo "This script must be run as root" 1>&2
   exit 1
fi
# ...

SVN auto add and delete

Posted by Dan Sosedoff on January 16, 2009

Bored of manually delete and add files to project? Ok, me too :) So, i decided to make a simple command to work with command “svn commit”. All-in-one solution is:

#!/bin/bash
 
echo "processing files to add..."
svn status | grep “^?” | sed -r ’s/^\?[ ]+//| xargs -r svn add
 
echo "processing files to delete..."
svn status | grep “^!| sed -r ’s/^\![ ]+//| xargs -r svn delete
 
echo "processing commit..."
svn commit

Download bash script – http://files.sosedoff.com/62c99246/