Tikejhya: Ashish Nepal

Knowledgebase

Category: BASH (Page 1 of 7)

Bash format option [disable autocommenting]

Your ads will be inserted here by

Easy Plugin for AdSense.

Please go to the plugin admin page to
Paste your ad code OR
Suppress this ad slot.

#Bash format option [disable autocommenting]
:set formatoptions-=cro

percona sync simplified to run over huge database

Your ads will be inserted here by

Easy Plugin for AdSense.

Please go to the plugin admin page to
Paste your ad code OR
Suppress this ad slot.

#!/bin/bash
if [ $# -lt 3 ] ; then
echo "USAGE $0 slavename db_name type"
echo "slavename [e.g db2, db3, db4 ]"
echo "db_name [e.g db1 db2 etc... calm down one at a time]"
echo "type [xxl_table_array xl_table_array unwanted_table all_small]"
exit
fi

LOCKFILE="/tmp/pt_sync.lock"
if [ -e "${LOCKFILE}" ]; then
echo "Already running."
exit 99

else

echo $! > "${LOCKFILE}"
chmod 644 "${LOCKFILE}"
slavehost=$1
backup_dir="./$1-table-sync-`date +%Y-%m-%d-%H-%M`"
mkdir -p $backup_dir
table_size=$3
db=$2

get_list_of_tables()

{
case "$table_size" in
xxl_tables)
tbls="xxl_table_name"
;;
xl_tables)
tbls="xl_table_1 xl_table_2"
;;
large_tables)
tbls="large_table_1 large_table_2"
;;
unwanted_table)
tbls="temp_table1 temp_table2"
;;
all_small)
allsmalltable_sync
;;
*)
echo "chow chow; did you miss anything?"
;;
esac
}

execute_query()
{
for tb in $tbls; do
pt-table-sync h=db1,D=$db,t=$tb h=$slavehost --user=username --password=password --no-check-slave --print >> $backup_dir/$db-$tb.log

done
}
allsmalltable_sync()
{
tbls="$(mysql -Bse "use $db; show tables;" | grep -v 'all_but_shit_tables\|all_but_shit_table_2')"
for tb in $tbls; do
pt-table-sync h=db1,D=$db,t=$tb h=$slavehost --user=username --password=password --no-check-slave --print >> $backup_dir/$db-$tb.log

done
}

get_list_of_tables $2
execute_query
rm -f "${LOCKFILE}"

fi

AWS Automated Snapshot

Your ads will be inserted here by

Easy Plugin for AdSense.

Please go to the plugin admin page to
Paste your ad code OR
Suppress this ad slot.

#!/bin/bash

# Volumelist => volume_id:someidentifiername:retaintion_period
volume_list='

vol-01010101:ashishnepal.com:3
vol-01010102:tikejhya.com:3
vol-01010103:tikejhyaroot.com:3
vol-01010104:somerandombox.com:3

'

snapshot_volumes() {
for volume in $volume_list; do

volume_id=$(echo "$volume" | cut -d':' -f1)
host_name=$(echo "$volume" | cut -d':' -f2)
retention_days=$(echo "$volume" | cut -d':' -f3)
retention_date_in_seconds=$(date +%s --date "$retention_days days ago")
snapshot_description=$(date +%Y-%m-%d)

snapshot_id=$(aws ec2 create-snapshot --output=text --description $host_name-$snapshot_description --volume-id $volume_id --query SnapshotId)
aws ec2 create-tags --resource $snapshot_id --tags Key=CreatedBy,Value=AutomatedBackup

done
}

cleanup_snapshots() {
for volume in $volume_list; do

volume_id=$(echo "$volume" | cut -d':' -f1)
host_name=$(echo "$volume" | cut -d':' -f2)
retention_days=$(echo "$volume" | cut -d':' -f3)
retention_date_in_seconds=$(date +%s --date "$retention_days days ago")

snapshot_list=$(aws ec2 describe-snapshots --output=text --filters "Name=volume-id,Values=$volume_id" "Name=tag:CreatedBy,Values=AutomatedBackup" --query Snapshots[].SnapshotId)
for snapshot in $snapshot_list; do

snapshot_date=$(aws ec2 describe-snapshots --output=text --snapshot-ids $snapshot --query Snapshots[].StartTime | awk -F "T" '{printf "%s\n", $1}')
snapshot_date_in_seconds=$(date "--date=$snapshot_date" +%s)
snapshot_description=$(aws ec2 describe-snapshots --snapshot-id $snapshot --query Snapshots[].Description)

if (( $snapshot_date_in_seconds <= $retention_date_in_seconds )); then aws ec2 delete-snapshot --snapshot-id $snapshot echo "aws ec2 delete-snapshot --snapshot-id $snapshot" else echo "Nothing to delete $snapshot" fi done done } cleanup_snapshots snapshot_volumes

diff between two files

diff -u file1 file2 | grep -E “^\+”

Bash: occurence count

This is simple bash onliner to count number of same digit match:

I needed to count 1,10,100,1000 occurrence in a file

for num in {1..10}; do echo “$num Digit”; cat /tmp/tt-28 | grep -E -w [0-9]{$num}

cat /tmp/tt-28
1
2
223
4
5
5

so in the given example you find, output as 5 1 Digit and 1 3 digit rest will be shown as 0.

[root@76 ~]# for num in {1..10}; do echo “$num Digit”; cat /tmp/tt-28 | grep -E -w [0-9]{$num} | wc -l ; done
1 Digit
5
2 Digit
0
3 Digit
1

crontab or shell script trailing with ^M

you might occasionally endup with scripts or cronjobs with trailing ^M and would like to quickly check if its so.

I use:

crontab -l | cat -tve

dstat

Options:

dstat [-afv] [options..] [delay [count]]

Dstat allows you to view all of your system resources instantly, you can eg. compare disk usage in combination with interrupts from your IDE controller, or compare the network bandwidth numbers directly with the disk throughput (in the same interval).

Best usages
dstat -cdgilmnprstTy --socket

find uninterruptible or dreaded process

D uninterruptible sleep (usually causing IO)

watch -n 1 "(ps aux | awk '$8 ~ /D/ { print $0 }')"

bash sort decending order

sort --version-sort

Or you can also use:

-b, –ignore-leading-blanks
ignore leading blanks

-g, –general-numeric-sort
compare according to general numerical value

-n, –numeric-sort
compare according to string numerical value

sort -g

delete history & exit

unset HISTFILE && exit

cat /dev/null > ~/.bash_history && history -c && exit

Page 1 of 7

Powered by WordPress & Theme by Anders Norén