dbawiki

Find all database-related files from the controlfile
List files using find without recursively descending into subdirectories
Print shell commands as they are being run
How to assign a certain keyboard shortcut to paste specific item
A poor man's 'expect'
Remove a directory from the PATH variable
Random password generator for SQL*Plus or PL/SQL
Inventory commands for AIX
Monitoring commands for AIX
AIX hacmp (PowerHA) startup and shutdown scripts
Other userful cluster commands
cat a local file to the end of a file on a remote host over ssh
Countdown timer loop
Sort a comma separated list in shell
Generate a random sequence of characters (maybe for a password)
How to automatically download and process email attachments from Gmail
Reset terminal to a sane state if it has got messed up
The underused xargs command!
Watch a directory and automatically perform an action when new files come into it
Top 10 (or 20) thread count consumers in AIX
How much memory is being used on AIX?
Top memory consumers in AIX
Top 10 memory-consuming processes
Show paged memory hogs on AIX
Top 10 processes in order of I/O usage
Top 10 CPU consuming processes
How can I either encrypt or render my shell script unreadable?
Date arithmetic
Out of memory saving lines for undo - try using ed
Restart a program / service / daemon if it has not been seen for a while
Start / Stop NFS mounts
Start / Stop / List Samba shares
Script called at node startup using a file on a shared cluster filesystem to start Oracle databases and listeners
Script called by previous one to start databases and / or listeners (supplied by Oracle Corp. many eons ago
Use sed to repeat characters
Convert between lower and upper case using ansi standard
Use sed to delete the first line of a file or command output printing from the second line onwards
Use sed to munge a controlfile into submission!
Find all files with lines containing a certain text
Find all files where a certain text does not exist
tar and gzip in one command using AIX
Use tar and gzip to copy files more efficiently across network between hosts
double hash and double percent in shell variables to trim off characters from variables
The best shell script ever. An example of how scripting should be done
Use Input Field Separator (IFS) to split a set of numbers into its constituent parts
Split a file into pieces
Strip out duplicate lines with this clever awk trick
split string (eg. filename) into separate variables using set
Centre align text on a line in bash shell
Right justify text (or repeat characters) on a line with leader dots!
cron jobs not submitted - daemon run out of child processes?
Find long-running processes with a cron job
Process command line arguments in shell
Carry XWindows settings across sessions
Cross-platform version of whoami
Set terminal title from command line
Remove blank lines and comments (also indented ones) from a file
Return elements of an array in Korn shell
A decent Unix Prompt
Simple arithmetic
Script encryption and passphrase protection
Virtual host configuration in Apache http.conf
Mount a website (or any other remote resource) locally using WebDav
Using Borg to backup to a remote server via sshfs
Use sshfs to mount a remote filesystem locally
Add current hostname to list of hosts on an xcat server
What is todays 'Day Of the Year' number?
Convert Julian day numbers to dates
Send crontab job output to a date formatted log file
Edit crontab file without crontab -e
Use shell to convert a number in scientific notation to normal
Check for jobs running longer that 24 hours
Global search and replace with perl (restricted to list of files provided by Unix find command)
perl function to make filenames lower case
From the management server, search the TNS listener port for each database on a server and make an inline substitution in the ITM config files!
Run a job from cron every Nth day of the month
ps listing does not show start time after 24 hours
ps -ef cuts off args cmd column on Solaris
Remove blank / empty lines from vi
Right pad a variable
Connect to a Windows server from Linux using rdesktop
Reset your password bypassing password rules
Sum the sizes of all files of an ls listing
Mount an iso image under Linux
How many processors on the machine?
Quick, simple, understandable example of how to use RRDTool
Use expect to respond automatically to interactive programs
Use expect to allow file copy with scp (if ssh keys are not an option)
Send logging of a shell script to file from inside the script
Log output of a shell script to both terminal and a file using a named pipes
Another example of how to set up simultaneous output to both terminal and a logfile using file descriptors
Execute an SQL file from a shell and email the results to a user
RedHat root filesystem has gone read-only
Kill all processes for a user
isRGHere
Find cluster name for current node of an hacmp cluster
AIX: add a user to a group
Find swear words in scripts by comparing it to a list found on the internet
grep -p (for paragraph) works on AIX but not on Linux or Solaris
debug/redirect log of a shell
Different ways of Iteration in korn shell with a while loop
Is it a directory or a filesystem mountpoint?
Filesystem 100% full, what's taking up all the space?
df -g shows filesystem full but du -gsx shows it as not full!
Capitalise the first letter of a word in Korn Shell
Remove / rename a directory / file containing weird control characters
Run a local script on a remote host
Get a list of running instances - but only those started by the current user
Return a list of Oracle databases running on a list of remote servers
Clever trick to check whether SQL output needs sending to someone
Check RMAN logfiles for errors from cron every day
Standard getopts
An alternative to getopts
getopts - another way - found in /etc/init.d/functions (daemon function) in Fedora
getopts - yet another way - found in adstrtal.sh (middleware start script for EBS)
getopts - and another in adautocfg.sh
getopts - ok, this is the last one (from ${OMS_HOME}/OMSPatcher/wlskeys/createkeys.sh
Run a script on multiple servers
Get a stack trace a running process on Linux
Get a stack trace a running process on AIX
Trace a running process on Linux
Trace a running process on AIX
Find hardware information on AIX
How can a shell script find out what directory it is in?
Array processing
Send an email by talking directly to an smtp server
Send an email by talking directly to an smtp server via file descriptor (no telnet! this time), adding authentication
Send an html file to an email recipient from a Unox server using sendmail
rsync examples
Handy aliases
How to configure SSH with public/private keys
Use SSH config file
A decent sed tutorial
A decent korn/bash shell tutorial
trap
DNS not working
Create new image with kvm
rpm / yum commands
Install rpmforge repository
Install rpmfusion repository
config file for yum
Setup Oracle Enterprise Linux (RedHat) with yum server
To change to static IP address (Raspberry Pi)
Troubleshoot wireless network problems
To change to static IP address (Redhat/CentOS)
Enable processes / services to start at boot time
Run a command on lots of servers in parallel
Download a gzip file and pipe it into tar
Check in a script to make sure it is run only by root
Set terminal to use Backspace key to erase previous character instead of Control-H
Play with the terminal settings and reset them again either side of requesting a password
Reset terminal to "sane" characteristics
Install OpenOffice on RedHat Enterprise when yum install doesn't!
What does this do?
List of Special Characters and what they mean

Bourne shell idioms - billharlan.com

Find all database-related files from the controlfile

Assumes a backup to trace of the controlfile is available

perl -ne 'print unless 1../Set #2. RESETLOGS/' "${CTRLFILE}"|grep -v SETCONFIG|grep -E "\'.+\/"|awk -F"'" '{print "rm -f "$2}'

List files using find without recursively descending into subdirectories

Stack Exchange

The idiomatic way is

find . ! -name . -prune -name "filestosearch*"

The key is the placement on the line of the -prune parameter

set lines 1000
col stmt for a500
select 'find '||destination||' -mtime +28 -prune -type f -ls' stmt from v$archive_dest where destination is not null;
select 'find '||destination||' -mtime +28 -prune -type f -exec rm {} \;' stmt from v$archive_dest where destination is not null;

This searches for and deletes all files in the archivelog destination older than 28 days without descending into the FRA subdirectory! It does no harm as the backup retention is only guaranteed for 28 days anyway. So all this does is clean up files that may have been lying around after a clone startup or before a db was switched to noarchivelog mode.

Print shell commands as they are being run

tackoverflow

Note: Support for local - was added in Bash 4.4.

function print_and_run_cmd
{
    local PS4='Running cmd: '
    local -
    set -o xtrace

    "$@"
}

How to assign a certain keyboard shortcut to paste specific item

If installed, xsel and xvkbd can be used

xsel -ib <<< 'Your string goes here'

xsel -ib < your-file.txt

xvkbd -file - <<< 'Your string goes here'

xvkbd -file your-file.txt

Assigning it to a Key combination in the window manager might require enclosing everything in quotes

bash -c "xsel -ib <<< 'my string'"

A poor man's 'expect'

Logging into, and using, telnet in batch mode, having the commands piped to it from a shell script

Or, using a shell's own logfile to interrogate what it is, itself, doing!

Use something like this

./telnet_prompts.ksh | telnet >telnet.log
</code
This runs the shell and pipes its output to telnet. All the output is sent to a logfile. This logfile will be read by the shell script!

telnet_prompts.ksh looks like this
<code>
#!ksh
host="127.0.0.1"
port=23
login="someuser"
passwd="somepass"
cmd="ls"
timeout=3
file="telnet.log"
prompt="$"

echo open ${host} ${port}
sleep 1
tout=${timeout}
while [ "${tout}" -ge 0 ]
do
    if tail -1 "${file}" 2>/dev/null | egrep -e "login:" > /dev/null
    then
        echo "${login}"
        sleep 1
        tout=-5
        continue
    else
        sleep 1
        tout=`expr ${tout} - 1`
    fi
done

if [ "${tout}" -ne "-5" ]; then
  exit 1
fi

tout=${timeout}
while [ "${tout}" -ge 0 ]
do
    if tail -1 "${file}" 2>/dev/null | egrep -e "Password:" > /dev/null
    then
        echo "${passwd}"
        sleep 1
        tout=-5
        continue
    else
      if tail -1 "${file}" 2>/dev/null | egrep -e "${prompt}" > /dev/null
      then
        tout=-5
      else
        sleep 1
        tout=`expr ${tout} - 1`
      fi
    fi
done

if [ "${tout}" -ne "-5" ]; then
  exit 1
fi

> ${file}

echo ${cmd}
sleep 1
echo exit

Remove a directory from the PATH variable

Change all the colons into carriage returns, exclude the lines containing the directory and rejoin the lines with a colon!

export PATH=$(echo $PATH | tr ":" "\n" | grep -v '/usr/local/bin' | xargs | tr ' ' ':')

Random password generator for SQL*Plus or PL/SQL

select dbms_random.string('a',15)||'#'||dbms_random.string('x',16) from dual;

Inventory commands for AIX

From https://www.netiq.com/documentation/platespin-recon-42/recon_user/data/b17df8u6.html

Command  Description
rpm      Get a list of installed programs.
ps       Get service (daemon) information.
mount    Get list of mounted volumes.
df       Get size information for mounted volumes.
lslv     Get SerialNumber and Label for mounted volumes.
hostname Get hostname.
namerslv Get DNS domain name.
getconf  Determine whether a 64-bit kernel.
uname    Determine OS release version.
ifconfig Get configured network adapters and their IP address.
netstat  Get MAC address.
entstat  Get speed of network adapter.
prtconf  Get processor information and default gateway.
lspv     Get list of disks and their partitions.
bootinfo Get disk size.
lscfg    Get disk model, manufacturer and type and network adapter model.
lsitab   Get daemon information.
getconf  Get machine memory and bootloader information.
lsattr   Get machine serial number and model,hyperthreading details and subnet mask.

Monitoring commands for AIX

Command  Description
vmstat   Get Available Memory in Bytes
ifconfig Get NIC information
entstat  Get Network Packets, Bytes Send/Received
mpstat   Get Percentage processor time
iostat   Get Disk transfer per second, disk idle, disk byte per second
sar      Get Disk Queue length, Percentage processor used
lspv     Get Disk partitioned or Unpartitioned space, disk list
bootinfo Get disk size
df       Get the volume level information
oslevel  Get the version of AIX

AIX hacmp (PowerHA) startup and shutdown scripts

/usr/es/sbin/cluster/utilities/cllsserv -h -c

gives something like

#Name:Start_script:Stop_script:start_mode:cpu_usage_monitor:process_to_monitor_cpu:cpu_usage_monitor_interval:
hn481_app:/ux/hacmp/bin/app_start.ksh HN481:/ux/hacmp/bin/app_stop.ksh HN481:background:no::10:

The app_start.sh script referenced above

#!/usr/bin/ksh
#-----------------------------------------------------------------
#
# start_applications
#
#-----------------------------------------------------------------
#
# Customer     : CLN
# Environment  : ALL
# version      : 3.0
# date         : 11/2008
# syntax       : app_start.ksh
# description  : This script is called by HACMP to start the HA
#                applications.  It reads ../hacmp/rc2.d directory
#                and runs all script beginning whith S*
# remarks      :
# history      :
#
#==================================================================
FILENAME=`basename $0`

if [ $# -ne 1 ]
then
  echo "$FILENAME require a parameter"
  exit 1
fi

NODE=$1

if [ ! -d /ux/hacmp/$NODE ]
then
  echo "!!! No /ux/hacmp/$NODE directory available !!!"
  exit 1
fi

export HACMP_RG_NAME=$NODE

APPBIN=/ux/hacmp/$NODE/bin
APPLOG=/uxlog/hacmp/$NODE
APPFIL=/ux/hacmp/$NODE/file
APPTMP=/ux/hacmp/$NODE/tmp
APPRC2=/ux/hacmp/$NODE/rc2.d

if [ ! -d $APPLOG ]
then
  for dir in app oracle samba
  do
    mkdir -p $APPLOG/$dir
  done
fi

LOGFILE="$APPLOG/$FILENAME.`date +%d%m%y`.`date +%H%M`.log"
exec 1>$LOGFILE 2>&1


   DATE=`date '+%Y-%m-%d_%H:%M:%S'`
   echo '\n###'
   echo "### Start of $0 - $DATE"

for f in $APPRC2/S*
        {
                if [ -s ${f} ]
                then
   DATE=`date '+%Y-%m-%d_%H:%M:%S'`
   echo '\n###'
   echo "### Start of $f - $DATE"
   echo '###\n'

            /usr/bin/ksh ${f} start
            rc=$?

   DATE=`date '+%Y-%m-%d_%H:%M:%S'`
   echo '\n###'
   echo "### End of $f - RC=$rc - $DATE"
   echo '###\n'
                fi
        }

   DATE=`date '+%Y-%m-%d_%H:%M:%S'`
   echo '\n###'
   echo "### End of $0 - $DATE"


exec 1<&-

exit 0

Other userful cluster commands

/usr/es/sbin/cluster/utilities/cllsres

APPLICATIONS="hn481_app"
FILESYSTEM=""
FORCED_VARYON="false"
FSCHECK_TOOL="fsck"
FS_BEFORE_IPADDR="true"
RECOVERY_METHOD="sequential"
SERVICE_LABEL="hn481"
SSA_DISK_FENCING="false"
VG_AUTO_IMPORT="false"
VOLUME_GROUP="ora1dwhtvg ora1prgvg app1prgvg app1tstvg ora1tstvg ora1tstarchvg hn481vg"
USERDEFINED_RESOURCES=""

/usr/es/sbin/cluster/events/cmd/cllsev -h -c

#name:cmd:desc
acquire_service_addr:/usr/es/sbin/cluster/events/acquire_service_addr:Script run when a service address is acquired on a node.
acquire_takeover_addr:/usr/es/sbin/cluster/events/acquire_takeover_addr:Script run when a service address is acquired on a node.
admin_op:/usr/es/sbin/cluster/events/admin_op:Script run when an administrator initiates a cluster operation.
cluster_ffdc:/usr/es/sbin/cluster/events/cl_ffdc:Script run to collect first failure data capture
cluster_notify:/usr/es/sbin/cluster/events/cluster_notify:Script run when SystemMirror detects a cluster configuration problem
config_too_long:/usr/es/sbin/cluster/events/config_too_long:Script run when a cluster event has been running for a long time.
event_error:/usr/es/sbin/cluster/events/event_error:Script run when there is a unrecoverable error while running a cluster event.
external_resource_state_change:/usr/es/sbin/cluster/events/external_resource_state_change:Scripts run when a cluster resource encounters a recoverable error
external_resource_state_change_complete:/usr/es/sbin/cluster/events/external_resource_state_change_complete:Scripts run when a cluster resource encounters a recoverable error
fail_interface:/usr/es/sbin/cluster/events/fail_interface:Script run after a network interface has failed.
forced_down_too_long:/usr/es/sbin/cluster/events/forced_down_too_long:Script run when a cluster node has been in the unmanaged state for some time
get_disk_vg_fs:/usr/es/sbin/cluster/events/get_disk_vg_fs:Script run to acquire disks, varyon volume groups, and mount filesystems.
intersite_fallover_prevented:/usr/es/sbin/cluster/events/intersite_fallover_prevented:Script run when a cluster resource is not moved across a site boundary
join_interface:/usr/es/sbin/cluster/events/join_interface:Script run after a network interface has recovered.
network_down:/usr/es/sbin/cluster/events/network_down:Script run when a network has failed.
network_down_complete:/usr/es/sbin/cluster/events/network_down_complete:Script run after the network_down script has successfully completed.
network_stable:/usr/es/sbin/cluster/events/network_stable:Script run when a network is stable.
network_unstable:/usr/es/sbin/cluster/events/network_unstable:Script run when a network is continuously changing state.
network_up:/usr/es/sbin/cluster/events/network_up:Script run after a network has become active.
network_up_complete:/usr/es/sbin/cluster/events/network_up_complete:Script run after the network_up script has successfully completed.
node_down:/usr/es/sbin/cluster/events/node_down:Script run when cluster services are stopped or a node has failed.
node_down_complete:/usr/es/sbin/cluster/events/node_down_complete:Script run after the node_down script has successfully completed.
node_down_local:/usr/es/sbin/cluster/events/node_down_local:Script run when it is the local node which is leaving the cluster.
node_down_local_complete:/usr/es/sbin/cluster/events/node_down_local_complete:Script run after the node_down_local script has successfully completed.
node_down_remote:/usr/es/sbin/cluster/events/node_down_remote:Script run when it is a remote node which is leaving the cluster.
node_down_remote_complete:/usr/es/sbin/cluster/events/node_down_remote_complete:Script run after the node_down_remote script has successfully completed.
node_up:/usr/es/sbin/cluster/events/node_up:Script run when a node is attempting to join the cluster.
node_up_complete:/usr/es/sbin/cluster/events/node_up_complete:Script run after the node_up script has successfully completed.
node_up_local:/usr/es/sbin/cluster/events/node_up_local:Script run when it is the local node which is joining the cluster.
node_up_local_complete:/usr/es/sbin/cluster/events/node_up_local_complete:Script run after the node_up_local script has successfully completed.
node_up_remote:/usr/es/sbin/cluster/events/node_up_remote:Script run when it is a remote node which is joining the cluster.
node_up_remote_complete:/usr/es/sbin/cluster/events/node_up_remote_complete:Script run after the node_up_remote script has successfully completed.
reconfig_configuration_complete:/usr/es/sbin/cluster/events/reconfig_configuration_complete:Script run while processing a dynamic configuration change.
reconfig_resource_acquire:/usr/es/sbin/cluster/events/reconfig_resource_acquire:Script run while processing a dynamic configuration change.
reconfig_resource_acquire_fence:/usr/es/sbin/cluster/events/reconfig_resource_acquire_fence:Script run while processing a dynamic configuration change.
reconfig_resource_acquire_secondary:/usr/es/sbin/cluster/events/reconfig_resource_acquire_secondary:Script run while processing a dynamic configuration change.
reconfig_resource_complete:/usr/es/sbin/cluster/events/reconfig_resource_complete:Script run while processing a dynamic configuration change.
reconfig_resource_complete_secondary:/usr/es/sbin/cluster/events/reconfig_resource_complete_secondary:Script run while processing a dynamic configuration change.
reconfig_resource_release:/usr/es/sbin/cluster/events/reconfig_resource_release:Script run while processing a dynamic configuration change.
reconfig_resource_release_fence:/usr/es/sbin/cluster/events/reconfig_resource_release_fence:Script run while processing a dynamic configuration change.
reconfig_resource_release_primary:/usr/es/sbin/cluster/events/reconfig_resource_release_primary:Script run while processing a dynamic configuration change.
reconfig_resource_release_secondary:/usr/es/sbin/cluster/events/reconfig_resource_release_secondary:Script run while processing a dynamic configuration change.
reconfig_topology_complete:/usr/es/sbin/cluster/events/reconfig_topology_complete:Script run while processing a dynamic configuration change.
reconfig_topology_start:/usr/es/sbin/cluster/events/reconfig_topology_start:Script run while processing a dynamic configuration change.
release_service_addr:/usr/es/sbin/cluster/events/release_service_addr:Script run when a service address is released from a node.
release_takeover_addr:/usr/es/sbin/cluster/events/release_takeover_addr:Script run when a service address is released from a node.
release_vg_fs:/usr/es/sbin/cluster/events/release_vg_fs:Script run to unmount filesystems and varyoff volume groups.
rep_disk_notify:/usr/es/sbin/cluster/events/rep_disk_notify:Script run when a repository disk has failed
resource_group_in_error:/usr/es/sbin/cluster/events/resource_group_in_error:Script run when a resource group goes to error state
resource_state_change:/usr/es/sbin/cluster/events/resource_state_change:Scripts run when a cluster resource encounters a recoverable error or the user moves resource groups.
resource_state_change_complete:/usr/es/sbin/cluster/events/resource_state_change_complete:Scripts run when a cluster resource encounters a recoverable error or the user moves resource groups.
rg_move:/usr/es/sbin/cluster/events/rg_move:Scripts run when SystemMirror moves resources and resource groups.
rg_move_acquire:/usr/es/sbin/cluster/events/rg_move_acquire:Scripts run when SystemMirror moves resources and resource groups.
rg_move_complete:/usr/es/sbin/cluster/events/rg_move_complete:Scripts run when SystemMirror moves resources and resource groups.
rg_move_fence:/usr/es/sbin/cluster/events/rg_move_fence:Scripts run when SystemMirror moves resources and resource groups.
rg_move_release:/usr/es/sbin/cluster/events/rg_move_release:Scripts run when SystemMirror moves resources and resource groups.
server_down:/usr/es/sbin/cluster/events/server_down:Script run when an application has failed.
server_down_complete:/usr/es/sbin/cluster/events/server_down_complete:Script run when an application has failed.
server_restart:/usr/es/sbin/cluster/events/server_restart:Script run to restart an application.
server_restart_complete:/usr/es/sbin/cluster/events/server_restart_complete:Script run to restart an application.
site_down:/usr/es/sbin/cluster/events/site_down:Script run when all nodes in a site are down.
site_down_complete:/usr/es/sbin/cluster/events/site_down_complete:Script run after the site_down script has successfully completed.
site_down_local:/usr/es/sbin/cluster/events/site_down_local:Script run when it is the local site which is leaving the cluster.
site_down_local_complete:/usr/es/sbin/cluster/events/site_down_local_complete:Script run after the site_down_local script has successfully completed.
site_down_remote:/usr/es/sbin/cluster/events/site_down_remote:Script run when it is a remote site which is leaving the cluster.
site_down_remote_complete:/usr/es/sbin/cluster/events/site_down_remote_complete:Script run after the site_down_remote script has successfully completed.
site_isolation:/usr/es/sbin/cluster/events/site_isolation:Scripts run when SystemMirror detects a site isolation or merge condition.
site_isolation_complete:/usr/es/sbin/cluster/events/site_isolation_complete:Script run after a site_isolation event completes.
site_merge:/usr/es/sbin/cluster/events/site_merge:Scripts run when SystemMirror detects a site isolation or merge condition.
site_merge_complete:/usr/es/sbin/cluster/events/site_merge_complete:Script run after a site_merge event completes.
site_up:/usr/es/sbin/cluster/events/site_up:Script run when cluster services are started on the first node in a site.
site_up_complete:/usr/es/sbin/cluster/events/site_up_complete:Script run after the site_up script has successfully completed.
site_up_local:/usr/es/sbin/cluster/events/site_up_local:Script run when it is the local site which is joining the cluster.
site_up_local_complete:/usr/es/sbin/cluster/events/site_up_local_complete:Script run after the site_up_local script has successfully completed.
site_up_remote:/usr/es/sbin/cluster/events/site_up_remote:Script run when it is a remote site which is joining the cluster.
site_up_remote_complete:/usr/es/sbin/cluster/events/site_up_remote_complete:Script run after the site_up_remote script has successfully completed.
split_merge_prompt:/usr/es/sbin/cluster/utilities/cl_sm_prompt:Script to prompt the operator for manual choice on split or merge
start_server:/usr/es/sbin/cluster/events/start_server:Script run to start an application.
start_udresource:/usr/es/sbin/cluster/events/start_udresource:Script run to start application servers.
stop_server:/usr/es/sbin/cluster/events/stop_server:Script run to start an application.
stop_udresource:/usr/es/sbin/cluster/events/stop_udresource:Script run to stop application servers.
swap_adapter:/usr/es/sbin/cluster/events/swap_adapter:Script run to swap IP Addresses between two network interfaces.
swap_adapter_complete:/usr/es/sbin/cluster/events/swap_adapter_complete:Script run after the swap_adapter script has successfully completed

cat a local file to the end of a file on a remote host over ssh

Nice trick but not really a trick! It saves having to scp the file over and then login and cat it on the remote host.

The contents of the local file are catted to STDOUT which is piped over ssh to a waiting cat which appends the incoming STDIN to the specified file.

cat id_rsa.pub | ssh oracle@hn5306 'cat >> .ssh/authorized_keys'

Countdown timer loop

Two examples from /usr/es/sbin/cluster/etc/rc.cluster

#!/bin/ksh93

typeset i=3  # make sure init has time to start it

while (( i-- ))
do
    src_running=$(ps -e | awk '$NF == "srcmstr" { print $1; exit }')

    [[ -n "$src_running" ]] && break
done

if (( $i == 0 ))
then
    cl_echo 234 "\n$PROGNAME: srcmstr is not running.\n" $PROGNAME
    exit 1
fi

if [[ -n "$src_running" ]]
then
   cl_echo 235 "Checking for srcmstr active...\n"
   typeset -i i=10  # try ten times to contact it

    while (( i-- ))
    do
      lssrc -s inetd >/dev/null 2>&1 && break  # break out on success
      sleep 1  # otherwise wait a second and try again
      echo ".\c"
    done

    if (( i == 0 ))
    then
       cl_echo 236 "\n\nERROR: srcmstr is not accepting connections\n"
       exit 1
    fi

    cl_echo 237 "complete.\n"
fi

Sort a comma separated list in shell

How to sort comma separated values in shell

Not as easy as it sounds. Googling it shows many varied ways mostly using ugly label syntax.
In amongst all the chaff, I spotted this wheat!

echo "7, 15, 6, 2, -9" | sed -e $'s/,/\n/g' | sort -n | tr '\n' ',' | sed 's/,$//'
sed -e $'s/,/\n/g'  : For splitting string into lines by comma.
sort -n               : Then you can use sort by number
tr '\n' ','           : Convert newline separator back to comma.
sed 's/,$//'          : Removing trailing comma.

It's that dollar sign that makes it possible. Without it, it fails. I have no idea what it does and I can find no info on it!

Generate a random sequence of characters (maybe for a password)

echo $(< /dev/urandom tr -dc %A-Z-a-z-0-9 | head -c${1:-30})

As that one might generate a number as the first character, this one generates 7 characters, followed by a '#', followed by 8 numbers. Should satisfy most

ORAPW=$(echo "$(tr -dc A-Za-z</dev/urandom|head -c7)#$(tr -dc 0-9</dev/urandom|head -c8)")

How to automatically download and process email attachments from Gmail

Useful references

Eventually, after months of trying different things, came up with this combination of tools that worked very well.

Install tools

sudo dnf -y install fetchmail maildrop uudeview openssl

Set Gmail account up for POP and disable IMAP

Configure fetchmail

This is the part that talks to Gmail and downloads the emails to the local machine.

vi ~/.fetchmailrc

set postmaster "<username>"
#set daemon 600
poll pop.gmail.com with proto POP3
    user "[email protected]" there with password '<password>' is <username> here options ssl nokeep
    mda "/usr/bin/maildrop .mailfilter"
#    sslcertfile /etc/ssl/certs/ca-bundle.crt

If fetchmail complains about ssl certificates, try specifying wherein is with the sslcertfile line

Configure maildrop

maildirmake maildrop
maildirmake backup

vi ~/.mailfilter

LOGFILE = "/home/dbahawk/maildrop/maildrop.log"
DEFAULT="$HOME/maildrop"

# keep a copy
cc backup
`cd backup/new && rm -f dummy \`ls -t | sed -e 1,50d\``

if (/^To:.*getmail@...\.dnsalias\.org$/)
   {
        to Maildir/getmail/
   }

#if (/^To:.*dbahawk/)
#   {
#        to maildrop
#   }

if (/^To:.*dbahawk/)
    dotlock "auto.lock" {
    to "|uudeview -c -i"
}

Not really sure why this filter is not working properly. The cc works but the To: check doesn't. So I have a shell that runs separately to process the emails in the backup mail directory.

uudeview

This is a clever program that can extract attachments from emails and put them in a directory of you choosing.
This shell runs from cron at regular intervals to run the whole process

#!/usr/bin/ksh

# fetch any new mail from Gmail (uses .fetchmailrc)
# fetchmail log is $HOME/maildrop/maildrop.log
fetchmail -d0

# fetchmail uses maildrop as mda (uses .mailfilter) to save messages to a maildir.
# this is not working correctly but still copies the messages to backup

# pull out the attachments and send them to the incoming directory for the collector to process when it wakes up
uudeview -c -i -p ~/Downloads/dbahawk_forward backup/new/*

Reset terminal to a sane state if it has got messed up

If, for example, you have done a cat on an binary file, sometines the terminal gets screwed up. This can get it back.

<CTRL-J>stty sane<CTRL-J>

The use of CTRL + J (Linefeed) is in case the Return key has also stopped working.

The underused xargs command!

See man xargs

ls | xargs -t -I {} mv {} {}.old This command sequence renames all files in the current directory by adding .old to the end of each name. The -I flag tells the xargs
            command to insert each line of the ls directory listing where {} (braces) appear. If the current directory contains the files chap1, chap2, and chap3, this constructs
            the following commands:

            mv  chap1  chap1.old
            mv  chap2  chap2.old
            mv  chap3  chap3.old

Watch a directory and automatically perform an action when new files come into it

Using a watchdog program like this Python script can do the trick (Using Python's Watchdog to monitor changes to a directory)

import time
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler


class Watcher:
    DIRECTORY_TO_WATCH = "/path/to/my/directory"

    def __init__(self):
        self.observer = Observer()

    def run(self):
        event_handler = Handler()
        self.observer.schedule(event_handler, self.DIRECTORY_TO_WATCH, recursive=True)
        self.observer.start()
        try:
            while True:
                time.sleep(5)
        except:
            self.observer.stop()
            print "Error"

        self.observer.join()


class Handler(FileSystemEventHandler):

    @staticmethod
    def on_any_event(event):
        if event.is_directory:
            return None

        elif event.event_type == 'created':
            # Take any action here when a file is first created.
            print "Received created event - %s." % event.src_path

        elif event.event_type == 'modified':
            # Taken any action here when a file is modified.
            print "Received modified event - %s." % event.src_path


if __name__ == '__main__':
    w = Watcher()
    w.run()

Top 10 (or 20) thread count consumers in AIX

ps -ef -o thcount,user,pid,ppid,time,args|sort -nr|head -20

How much memory is being used on AIX?

svmon -O summary=basic,unit=GB

Unit: GB
--------------------------------------------------------------------------------------
               size       inuse        free         pin     virtual  available   mmode
memory       270.00      266.92        3.08        22.9      204.16       57.2     Ded
pg space       64.0        0.84

               work        pers        clnt       other
pin            14.7           0        0.04        8.18
in use       204.16           0        62.8

Top memory consumers in AIX

svmon -U -t 10 -O process=on -O sortentity=pgsp

Top 10 memory-consuming processes

ps aux | head -1; ps aux | sort -rn +3 | head

or, in order of real memory usage

ps vx | head -1; ps vx | grep -v PID | sort -rn +6 | head -10

Show paged memory hogs on AIX

Show top 10 processes using memory

svmon -Pt20 | perl -ne 'print if($.==2||$&&&!$s++);$.=0 if(/^-+$/)'

Top 10 processes in order of I/O usage

ps vx | head -1; ps vx | grep -v PID | sort -rn +4 | head -10

Top 10 CPU consuming processes

Info on High(100%) used cpu myth! "Chicken Little" myth - dba-oracle.com On AIX

ps aux | head -1; ps aux | sort -rn +2 | head -10

then

ps aeww <process_id>

or on Linux

top -b -n 1 | sed -e "1,6d" | head -11

gives something like…

   PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
 29185 oracle    20   0 6597120   2.2g  74936 S   6.2  7.0  11:18.92 java
 76486 oracle    20   0  129824   2168   1744 S   6.2  0.0 566:09.68 genorapw
     1 root      20   0  191692   4012   2112 S   0.0  0.0  29:21.87 systemd
     2 root      20   0       0      0      0 S   0.0  0.0   0:00.83 kthreadd
     4 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 kworker/0:0H
     6 root      20   0       0      0      0 S   0.0  0.0  24:24.49 ksoftirqd/0
     7 root      rt   0       0      0      0 S   0.0  0.0  52:02.11 migration/0
     8 root      20   0       0      0      0 S   0.0  0.0   0:00.00 rcu_bh
     9 root      20   0       0      0      0 S   0.0  0.0  94:06.86 rcu_sched
    10 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 lru-add-drain

Now we have the process 12385 showing 119% of cpu usage!

oracle    12385  12327 99 Aug17 ?        74-09:39:08 /cln/exp/ora_bin1/app/oracle/middleware/oracle_common/jdk/bin/java -server -Xms256M -Xmx1740M -XX:PermSize=128M -XX:MaxPermSize=768M -XX:CompileThreshold=8000 -XX:-DoEscapeAnalysis -XX:+UseCodeCacheFlushing -XX:ReservedCodeCacheSize=100M -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSClassUnloadingEnabled -Dweblogic.Name=EMGC_OMS1 -Djava.security.policy=/cln/exp/ora_bin1/app/oracle/middleware/wlserver/server/lib/weblogic.policy -Dweblogic.ProductionModeEnabled=true -Dweblogic.system.BootIdentityFile=/cln/exp/ora_bin1/app/oracle/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/data/nodemanager/boot.properties -Dweblogic.nodemanager.ServiceEnabled=true -Dweblogic.nmservice.RotationEnabled=true -Dweblogic.security.SSL.ignoreHostnameVerification=true -Dweblogic.ReverseDNSAllowed=false -DINSTANCE_HOME=/cln/exp/ora_bin1/app/oracle/gc_inst/em/EMGC_OMS1 -DORACLE_HOME=/cln/exp/ora_bin1/app/oracle/middleware -Ddomain.home=/cln/exp/ora_bin1/app/oracle/gc_inst/user_projects/domains/GCDomain -Djava.awt.headless=true -Ddomain.name=GCDomain -Doracle.sysman.util.logging.mode=dual_mode -Djbo.doconnectionpooling=true -Djbo.txn.disconnect_level=1 -Docm.repeater.home=/cln/exp/ora_bin1/app/oracle/middleware -Djbo.ampool.minavailablesize=1 -Djbo.ampool.timetolive=-1 -Djbo.load.components.lazily=true -Djbo.max.cursors=5 -Djbo.recyclethreshold=50 -Djbo.ampool.maxavailablesize=50 -Djavax.xml.bind.JAXBContext=com.sun.xml.bind.v2.ContextFactory -Djava.security.egd=file:///dev/./urandom -Dweblogic.debug.DebugWebAppSecurity=true -Dweb ogic.SSL.LoginTimeoutMillis=300000 -Djps.auth.debug=true -Djps.authz=ACC -Djps.combiner.optimize.lazyeval=true -Djps.combiner.optimize=true -Djps.subject.cache.key=5 -Djps.subject.cache.ttl=600000 -Doracle.apm.home=/cln/exp/ora_bin1/app/oracle/middleware/apm/ -DAPM_HELP_FILENAME=oesohwconfig.xml -Dweblogic.data.canTransferAnyFile=true -Dhttps.protocols=TLSv1,TLSv1.1,TLSv1.2 -DHTTPClient.retryNonIdempotentRequest=false -Dweblogic.security.SSL.minimumProtocolVersion=TLSv1 -Djava.endorsed.dirs=/cln/exp/ora_bin1/app/oracle/middleware/oracle_common/jdk/jre/lib/endorsed:/cln/exp/ora_bin1/app/oracle/middleware/oracle_common/modules/endorsed -Djava.protocol.handler.pkgs=oracle.mds.net.protocol -Dopss.version=12.1.3 -Digf.arisidbeans.carmlloc=/cln/exp/ora_bin1/app/oracle/gc_inst/user_projects/domains/GCDomain/config/fmwconfig/carml -Digf.arisidstack.home=/cln/exp/ora_bin1/app/oracle/gc_inst/user_projects/domains/GCDomain/config/fmwconfig/arisidprovider -Doracle.security.jps.config=/cln/exp/ora_bin1/app/oracle/gc_inst/user_projects/domains/GCDomain/config/fmwconfig/jps-config.xml -Doracle.deployed.app.dir=/cln/exp/ora_bin1/app/oracle/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/tmp/_WL_user -Doracle.deployed.app.ext=/- -Dweblogic.alternateTypesDirectory=/cln/exp/ora_bin1/app/oracle/middleware/oracle_common/modules/oracle.ossoiap_12.1.3,/cln/exp/ora_bin1/app/oracle/middleware/oracle_common/modules/oracle.oamprovider_12.1.3,/cln/exp/ora_bin1/app/oracle/middleware/oracle_common/modules/oracle.jps_12.1.3 -Doracle.mds.filestore.preferred= -Dadf.version=12.1.3 -Dweblogic.jdbc.remoteEnabled=false -Dcommon.components.home=/cln/exp/ora_bin1/app/oracle/middleware/oracle_common -Djrf.version=12.1.3 -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.Jdk14Logger -Ddomain.home=/cln/exp/ora_bin1/app/oracle/gc_inst/user_projects/domains/GCDomain -Doracle.server.config.dir=/cln/exp/ora_bin1/app/oracle/gc_inst/user_projects/domains/GCDomain/config/fmwconfig/servers/EMGC_OMS1 -Doracle.domain.config.dir=/cln/exp/ora_bin1/app/oracle/gc_inst/user_projects/domains/GCDomain/config/fmwconfig -Dohs.product.home=/cln/exp/ora_bin1/app/oracle/middleware/ohs -da -Dwls.home=/cln/exp/ora_bin1/app/oracle/middleware/wlserver/server -Dweblogic.home=/cln/exp/ora_bin1/app/oracle/middleware/wlserver/server -Djavax.management.builder.initial=weblogic.management.jmx.mbeanserver.WLSMBeanServerBuilder -Dxdo.server.config.dir=/cln/exp/ora_bin1/app/oracle/gc_inst/user_projects/domains/GCDomain/config/bipublisher -DXDO_FONT_DIR=/cln/exp/ora_bin1/app/oracle/middleware/bi/common/fonts -Dweblogic.management.server=https://hn1627.cln.be:7102 -Djava.util.logging.manager=oracle.core.ojdl.logging.ODLLogManager -Dweblogic.utils.cmm.lowertier.ServiceDisabled=true weblogic.Server

Digging deeper. Process 12385 is a java program (WebLogic server) and is the OMS component of Oracle Enterprise Manager (see -Dweblogic.Name=EMGC_OMS1). We can find which thread of the process is using most cpu with…

ps -eLo pid,ppid,tid,pcpu,comm |sort -k4| grep 12385

gives

 12385  12327  80591  0.0 java
 12385  12327  81262  0.0 java
 12385  12327  81265  0.0 java
 12385  12327  81297  0.0 java
 12385  12327  88307  0.0 java
 12385  12327   8833  0.0 java
 12385  12327  90626  0.0 java
 12385  12327  90627  0.0 java
 12385  12327  93238  0.0 java
 12385  12327  93314  0.0 java
 12385  12327  94330  0.0 java
 12385  12327  94454  0.0 java
 12385  12327  98821  0.0 java
 12385  12327  98824  0.0 java
 12385  12327  12391  0.1 java
 12385  12327  47759  0.2 java
 12385  12327  12399  0.3 java
 12385  12327  47962  0.5 java
 12385  12327  47963  0.5 java
 12385  12327  12388 17.3 java
 12385  12327  12389 17.3 java
 12385  12327  12390 17.3 java
 12385  12327  12387 17.4 java
 12385  12327  12392  2.2 java
 12385  12327  51148 24.0 java
 12385  12327 102611 40.4 java

so thread id 102611 of process 12385 is using 40% cpu As this is Enterprise Manager, instead of running

kill -3 12385

we run

$AGENT_HOME/oracle_common/jdk/bin/jstack -F 12385 >stacktrace_12385.txt

Look up the thread inside this trace file

Thread 28550: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise)
 - java.util.TimerThread.mainLoop() @bci=201, line=552 (Compiled frame)
 - java.util.TimerThread.run() @bci=1, line=505 (Interpreted frame)


Thread 93314: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise)
 - java.util.TimerThread.mainLoop() @bci=201, line=552 (Compiled frame)
 - java.util.TimerThread.run() @bci=1, line=505 (Interpreted frame)


Thread 102611: (state = BLOCKED)
 - java.lang.Long.valueOf(long) @bci=27, line=577 (Compiled frame)
 - oracle.adfinternal.view.faces.activedata.LongPollingChannelHandler._longPollEvents(javax.faces.context.ExternalContext, oracle.adfinternal.view.faces.activedata.PageDataUpdateManager, javax.servlet.http.HttpServletResponse, oracle.adfinternal.view.faces.activedata.AdsProcessor$ADSRequestParams) @bci=243, line=324 (Compiled frame)
 - oracle.adfinternal.view.faces.activedata.LongPollingChannelHandler.flushActiveData(javax.faces.context.ExternalContext, oracle.adfinternal.view.faces.activedata.PageDataUpdateManager, javax.servlet.http.HttpServletResponse, oracle.adfinternal.view.faces.activedata.AdsProcessor$ADSRequestParams) @bci=6, line=88 (Interpreted frame)
 - oracle.adfinternal.view.faces.activedata.AdsProcessor.flushActiveData(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=72, line=126 (Interpreted frame)
 - oracle.adfinternal.view.faces.activedata.ActiveDataFlusher.run() @bci=28, line=26 (Interpreted frame)
 - weblogic.work.j2ee.J2EEWorkManager$WorkWithListener.run() @bci=72, line=184 (Interpreted frame)
 - weblogic.work.ExecuteThread.execute(java.lang.Runnable) @bci=34, line=311 (Compiled frame)
 - weblogic.work.ExecuteThread.run() @bci=42, line=263 (Compiled frame)


Thread 119236: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise)
 - java.lang.Object.wait() @bci=2, line=503 (Compiled frame)
 - oracle.sysman.util.threadPoolManager.Worker.getWork() @bci=8, line=132 (Compiled frame)
 - oracle.sysman.util.threadPoolManager.WorkerThread.run() @bci=21, line=304 (Compiled frame)

Need to contact Oracle Support about this one…

Update: Before it was resolved, the system was shutdown and restarted as part of a DR exercise. Problem has gone away

How can I either encrypt or render my shell script unreadable?

Usage

$ ./shc -f random.sh

Once you run it your shell script, random.sh will get converted into this file:

-rwx-wx--x. 1 ramesh ramesh 11752 Mar 27 01:12 random.sh.x

Date arithmetic

In bash, take one hour off the current time

printf "current date: %(%m/%d/%Y -%H:%M:%S)T\n"
printf "date - 60min: %(%m/%d/%Y -%H:%M:%S)T\n" $(( $(printf "%(%s)T") - 60 * 60 ))

Out of memory saving lines for undo - try using ed

vi fails to open (semi) large files. Put this in .kshrc

export EXINIT="set ll=80000000"

Restart a program / service / daemon if it has not been seen for a while

Use the %s format of the date command to be able to to date arithmetic

#!/bin/ksh

MAX_DELTA=60

last_occurance_txt=$(/bin/grep sssd /var/log/messages | /bin/grep "Preauthentication failed" | /usr/bin/tail -1 | /bin/cut -c1-15)

if [ ! -z "$last_occurance_txt" ]
then
  last_occurance=$(/bin/date --date="$last_occurance_txt" +%s)
  now=$(/bin/date +%s)

  delta=$((now - last_occurance))
  if [ $delta -le $MAX_DELTA ]
  then
    /bin/date
    /sbin/service sssd restart
  fi
fi

Start / Stop NFS mounts

#!/bin/ksh

if test "x$GROUPNAME" == "x"
then
  echo "GROUPNAME does not exist script was not called by PowerHA"
  exit 16
fi

echo "NFS GROUPNAME=$GROUPNAME"

function _usage
{
  echo "usage: $1 {start|stop|list}"
  echo "   start : Mount the NFS for RG"
  echo "   stop : Unmount the NFS for RG"
}

if [ $# -ne 1 ] ; then
    _usage "$0"
    exit 1
fi

case "$1" in
  start) mount -t $GROUPNAME
         ;;
  stop)  umount -f -t $GROUPNAME
         ;;
  *)     _usage "$0"
         exit 1
         ;;
esac

Start / Stop / List Samba shares

#!/bin/ksh
DIRECT=$(dirname "$0")
HACMP_RG_NAME=$(echo $DIRECT | cut -d"/" -f4)

echo $HACMP_RG_NAME | grep -q HN
rc=$?
if [ $rc -ne 0 ]
then
  echo "Must be started with full path (/ux/hacmp/HN.../rc2.d/...)"
  exit
fi

HACMP_RG_NAME2=$((cd /ux/hacmp ; ls -d HN??? )| grep -vw $HACMP_RG_NAME)

LNN=`/usr/es/sbin/cluster/utilities/get_local_nodename|tr "[:lower:]" "[:upper:]"`
HACMP_REMOTE_HOST=$(/usr/es/sbin/cluster/utilities/cllsnode | grep "^Node" | awk '{print $2}' | grep -vw $(/usr/es/sbin/cluster/utilities/get_local_nodename)| tr "[:lower:]" "[:upper:]")
SMBCONF=/ux/hacmp/${HACMP_RG_NAME}/fil/smb.conf
SMBPID=/ux/hacmp/${HACMP_RG_NAME}/run/smbd-smb.conf.pid
NMBPID=/ux/hacmp/${HACMP_RG_NAME}/run/nmbd-smb.conf.pid
WINPID=/ux/hacmp/${HACMP_RG_NAME}/run/winbindd-smb.conf.pid

IP_SERVICE=$(grep -w ^interfaces $SMBCONF | cut -d"=" -f2 | awk '{print $1}' | cut -d"/" -f1)
INTERFACE=${IP_SERVICE}


function _usage
{
  echo "usage: $1 {start|stop|list}"
  echo "   start : start the SAMBA deamon"
  echo "   stop : start the SAMBA deamon"
  echo "   list  : list the SAMBA shares"
}

if [ $# -ne 1 ] ; then
    _usage "$0"
    exit 1
fi

case "$1" in
  [sS][tT][aA][rR][tT])
#       mkdev -l posix_aio0
       echo 'Starting smbd ...'
       /opt/pware64/sbin/smbd -s "$SMBCONF" -D
       echo 'Starting nmbd ...'
       /opt/pware64/sbin/nmbd -s "$SMBCONF" -D
       echo 'Starting winbindd ...'
       /opt/pware64/sbin/winbindd -s "$SMBCONF"
     ;;
  [sS][tT][oO][pP])
      # a nice kill of SAMBA processes
      for proc in $(cat $SMBPID $NMBPID $WINPID)
      do
        if [ -f $proc ]
        then
          echo "Sending SIGTERM to the process $proc ..."
          kill -TERM $proc > /dev/null 2>/dev/null
          sleep 1
        fi
      done
      ps -ef|grep pware|grep $HACMP_RG_NAME|grep -v grep|grep -v stop
      if [ $? == 0 ]
      then
      for proc in `ps -ef|grep pware|grep $HACMP_RG_NAME|grep -v grep|awk '{print $2}'`
      do
        kill -9 $proc
      done
      fi

#      if [ $LNN == $HACMP_REMOTE_HOST ]
#      then
#       /ux/hacmp/${HACMP_RG_NAME2}/init.d/samba.ksh stop;/ux/hacmp/${HACMP_RG_NAME2}/init.d/samba.ksh start
#      fi

     ;;
  [lL][iI][sS][tT])
        /opt/pware64/bin/smbclient -L $INTERFACE -N
     ;;
  info)
        echo "boot local      $LNN"
        echo "boot remote     $HACMP_REMOTE_HOST"
        echo "service local   $HACMP_RG_NAME"
        echo "service remote  $HACMP_RG_NAME2"
        echo "configuration   $SMBCONF"
        echo "IP_SERVICE      $IP_SERVICE"
        echo "INTERFACE       $INTERFACE"
     ;;
  *)
    _usage "$0"
    exit 1
    ;;
esac
exit 0

Script called at node startup using a file on a shared cluster filesystem to start Oracle databases and listeners

(0) nodb_121 oracle@hn5205:/home/oracle> sudo cat /ux/hacmp/init.d/oracle.ksh
#!/bin/ksh
DIRECT=$(dirname "$0")
# some configuration variables :

ORATAB=/ux/hacmp/${HACMP_RG_NAME}/file/oratab_${HACMP_RG_NAME}
LSNRDESC=/ux/hacmp/${HACMP_RG_NAME}/file/lsnrdesc_${HACMP_RG_NAME}

echo "ORATAB=$ORATAB"
echo "LSNRDESC=$LSNRDESC"

function _usage
{
  echo "usage: $1 {start|stop}"
  echo "   start : start Oracle DBs"
  echo "   stop : stop Oracle DBs"
}

if [ $# -ne 1 ] ; then
    _usage "$0"
    exit 1
fi

case "$1" in
  [sS][tT][aA][rR][tT])
       echo 'Starting Oracle ...'
       export ORATAB LSNRDESC
       echo 'Starting listener ...'
       for i in `sed -e 's/#.*$//g' -e '/^$/d' $LSNRDESC | awk -F":" '{print $1}'`
       do
                ORACLE_OWNER=`cat $LSNRDESC | awk -F":" '{if($1=="'$i'")print $4}'`
                export ORACLE_OWNER
                ORALOG=/uxlog/hacmp/${HACMP_RG_NAME}/oracle/start_${i}.log
                /ux/hacmp/init.d/rc.oracle.ksh start lsnr $i 1>$ORALOG 2>&1 &
       done
       wait
       sleep 10
       echo 'Starting DB ...'
       for i in `sed -e 's/#.*$//g' -e '/^$/d' $ORATAB | awk -F":" '{print $1}'`
       do
                ORACLE_OWNER=`cat $ORATAB | awk -F":" '{if($1=="'$i'")print $4}'`
                export ORACLE_OWNER
                ORALOG=/uxlog/hacmp/${HACMP_RG_NAME}/oracle/start_${i}.log
                /ux/hacmp/init.d/rc.oracle.ksh start db $i 1>$ORALOG 2>&1 &
       done
       wait
     ;;
  [sS][tT][oO][pP])
       echo 'Stopping Oracle ...'
       export ORATAB LSNRDESC
       echo 'Stopping DB ...'
       for i in `cat $ORATAB | awk -F":" '{print $1}'`
       do
                ORACLE_OWNER=`sed -e 's/#.*$//g' -e '/^$/d' $ORATAB | awk -F":" '{if($1=="'$i'")print $4}'`
                export ORACLE_OWNER
                ORALOG=/uxlog/hacmp/${HACMP_RG_NAME}/oracle/stop_${i}.log
                /ux/hacmp/init.d/rc.oracle.ksh stop db $i 1>$ORALOG 2>&1 &
       done
       wait
       echo 'Stopping listener ...'
       for i in `cat $LSNRDESC | awk -F":" '{print $1}'`
       do
                ORACLE_OWNER=`sed -e 's/#.*$//g' -e '/^$/d' $LSNRDESC | awk -F":" '{if($1=="'$i'")print $4}'`
                export ORACLE_OWNER
                ORALOG=/uxlog/hacmp/${HACMP_RG_NAME}/oracle/stop_${i}.log
                /ux/hacmp/init.d/rc.oracle.ksh stop lsnr $i 1>$ORALOG 2>&1 &
       done
       wait
     ;;
  *)
    _usage "$0"
    exit 1
    ;;
esac
exit 0

Script called by previous one to start databases and / or listeners (supplied by Oracle Corp. many eons ago

(0) nodb_121 oracle@hn5205:/home/oracle> sudo cat /ux/hacmp/init.d/rc.oracle.ksh
#!/bin/ksh
#########################################################################
#                                                                       #
#       Name:           rc.oracle                                       #
#                                                                       #
#       Version:        24/11/2008                                      #
#                                                                       #
#       Description:    This script starts and stops ORACLE V7, V8 and  #
#                       V9 databases as well as SQLNET listeners.       #
#                       It processes only the databases/listeners       #
#                       whose entry in the file $ORATAB/$LSNRDESC has   #
#                       the third field set to Y.                       #
#                       The original purpose of this script was to      #
#                       start and stop all databases and all listeners  #
#                       defined in the files $ORATAB and $LSNRDESC.     #
#                       It is however possible to start and stop only   #
#                       the databases or only the listeners, only one   #
#                       database or only one listener by specifying the #
#                       appropiated arguments.                          #
#                       A database is started with the "mount" option.  #
#                       If there is any tablespace in backup mode, its  #
#                       status is reset before the database is opened.  #
#                       For each database to be stopped, a "shutdown    #
#                       immediate" is performed in background. Before   #
#                       stopping the listeners, the script waits for    #
#                       the completion of the shutdowns. After a        #
#                       timeout of $TIME_BEF_ABORT seconds, it          #
#                       aborts the databases that are still in the      #
#                       shutdown process.                               #
#                       Variables ORATAB, LSNRDESC and TIME_BEF_ABORT   #
#                       have to be initialized in this script. For the  #
#                       value of $TIME_BEF_ABORT, take into account the #
#                       number of databases to shut and their size.     #
#                                                                       #
#       Arguments:      [start|stop] [db|lsnr] [DBSID|LSNRNAME]         #
#                                                                       #
#       Files:          $ORATAB (/etc/dbdesc for example),              #
#                       $LSNRDESC (/etc/lsnrdesc for example)           #
#                                                                       #
#       Scripts:                                                        #
#                                                                       #
#       Returns:        1 if file $ORATAB or file $LSNRDESC is          #
#                         inaccessible or if bad argument received.     #
#                       0 otherwise                                     #
#                                                                       #
#########################################################################
date

if [ -z "${ORATAB}" -o -z "${LSNRDESC}" ]
then
        echo "ORATAB or LSNRDESC not set"
        exit 1
fi
TIME_BEF_ABORT=360

if [ "$VERBOSE_LOGGING" = "high" ]
then
    set -x
fi

# oracle processes CLEARING
kill_oracle_process ()
{
if [ "$VERBOSE_LOGGING" = "high" ]
then
    set -x
fi
sleep 5
PROC_NAME="oracle$1"
ps -eo pid,args | awk '{if($2=="'$PROC_NAME'")print $1}' | while read PROC_NBR
do
    kill -9 $PROC_NBR
done
}

# oracle processes CHECKING
oracle_proc_running ()
{
if [ "$VERBOSE_LOGGING" = "high" ]
then
    set -x
fi
SID=$1
NUMBOP=$2
X=`ps -eo args | awk '$1 ~ /ora_.*_'$SID'$/' | wc -l`
if [ $X -ge $NUMBOP ]
then
    return 0  # ORACLE RUNS FOR SID = $1
else
    return 1  # ORACLE DOES NOT RUN FOR SID = $1
fi
}

case $1 in
    ''|'start') TASK="START";;
    'stop')     TASK="STOP";;
    *)          echo "Usage $0 [start|stop] [db|lsnr] [DBSID|LSNRNAME]";
                exit 1;;
esac
case $2 in
    '')       TASK=$TASK"ALL";;
    'db')     TASK=$TASK"DB";;
    'lsnr')   TASK=$TASK"LSNR";;
    *)        echo "Usage $0 [start|stop] [db|lsnr] [DBSID|LSNRNAME]";
              exit 1;;
esac
DLNAME="^.*:"
if [ "$TASK" != "STARTALL" -a "$TASK" != "STOPALL" -a "$3" != "" ]
then
    DLNAME="^${3}:"
fi

PATH=/usr/bin:/etc:/usr/sbin:/usr/bin/X11; export PATH
SAVE_PATH=$PATH; export SAVE_PATH
LIBPATH=/usr/lib:/usr/lib/X11; export LIBPATH
SAVE_LIBPATH=$LIBPATH; export SAVE_LIBPATH
AIXTHREAD_SCOPE=S; export AIXTHREAD_SCOPE

if [ "$TASK" = "STARTALL" -o "$TASK" = "STARTLSNR" ]
then
    if [ ! -r $LSNRDESC ]
    then
        echo "Can't access $LSNRDESC"
        exit 1
    fi
    echo "\nORACLE listeners startup\n************************"
    grep "^[a-z,A-Z,0-9]" $LSNRDESC | grep $DLNAME | ;
       awk -F":" '{if($3=="Y")print $1,$2}' | ;
       while read LSNR_NAME ORACLE_HOME
    do
        if [ ! -x $ORACLE_HOME/bin/lsnrctl ]
        then
            echo "Can't find or execute $ORACLE_HOME/bin/lsnrctl"
        else
            PATH=$ORACLE_HOME/bin:$SAVE_PATH
            if [ -d $ORACLE_HOME/lib32 ]
            then
                LIBPATH=$ORACLE_HOME/lib:$ORACLE_HOME/lib32:$SAVE_LIBPATH
            else
                LIBPATH=$ORACLE_HOME/lib:$SAVE_LIBPATH
            fi
            LD_LIBRARY_PATH=$LIBPATH
            TNS_ADMIN=$ORACLE_HOME/network/admin
            export ORACLE_HOME PATH LIBPATH LD_LIBRARY_PATH TNS_ADMIN
            if [ -r $ORACLE_HOME/ops/ogms_profile.sh ]
            then
                . $ORACLE_HOME/ops/ogms_profile.sh
            fi
            if [ "$LSNR_NAME" = "DBSNMP" ]
            then
                su $ORACLE_OWNER -c "lsnrctl dbsnmp_start"
            else
                su $ORACLE_OWNER -c "lsnrctl start $LSNR_NAME"
            fi
        fi
    done
    [ "$TASK" = "STARTALL" ] && sleep 10
fi

if [ "$TASK" = "STARTALL" -o "$TASK" = "STARTDB" ]
then
    if [ ! -r $ORATAB ]
    then
        echo "Can't access $ORATAB"
        exit 1
    fi
    echo "\nORACLE databases startup\n************************"
    grep "^[a-z,A-Z,0-9]" $ORATAB | grep $DLNAME | ;
       awk -F":" '{if($3=="Y")print $1,$2}' | ;
       while read ORACLE_SID ORACLE_HOME
    do
        PATH=$ORACLE_HOME/bin:$SAVE_PATH
        if [ -d $ORACLE_HOME/lib32 ]
        then
            LIBPATH=$ORACLE_HOME/lib:$ORACLE_HOME/lib32:$SAVE_LIBPATH
        else
            LIBPATH=$ORACLE_HOME/lib:$SAVE_LIBPATH
        fi
        LD_LIBRARY_PATH=$LIBPATH
        export ORACLE_HOME ORACLE_SID PATH LIBPATH LD_LIBRARY_PATH
        if [ -r $ORACLE_HOME/ops/ogms_profile.sh ]
        then
            . $ORACLE_HOME/ops/ogms_profile.sh
        fi
        export SQLDBA=""
        if [ -x $ORACLE_HOME/bin/svrmgrl ]
        then
            SQLDBA=svrmgrl
        elif [ -x $ORACLE_HOME/bin/sqlplus ]
        then
            SQLDBA="sqlplus /nolog"
        fi
        if [ "$SQLDBA" = "" ]
        then
            echo "Can't find or execute svrmgrl|sqlplus under $ORACLE_HOME/bin"
        elif [ ! -f $ORACLE_HOME/dbs/init${ORACLE_SID}.ora -a ! -f $ORACLE_HOME/dbs/spfile${ORACLE_SID}.ora ]
        then
            echo "Can't find init file for Database $ORACLE_SID"
        elif oracle_proc_running $ORACLE_SID 4
        then
            echo "ORACLE IS ALREADY RUNNING FOR SID : $ORACLE_SID"
        else
            echo "Clearing memory for database $ORACLE_SID"
            su $ORACLE_OWNER -c '${SQLDBA} <<EOED
connect / as sysdba;
shutdown abort;
disconnect;
exit;
EOED'
            echo "Mounting Oracle database $ORACLE_SID"
            su $ORACLE_OWNER -c '${SQLDBA} <<EOED
connect / as sysdba;
startup mount;
select * from v\$backup;
disconnect;
exit;
EOED' | tee /tmp/start${ORACLE_SID}.log | ;
            awk '{if($2=="ACTIVE")print $1}' | while read DFN
            do
                echo "alter database datafile $DFN END BACKUP;"
            done > /tmp/endbkp${ORACLE_SID}.sql
            if [ -s /tmp/endbkp${ORACLE_SID}.sql ]
            then
                echo "Resetting the status of datafile(s) found in backup mode"
                chmod 644 /tmp/endbkp${ORACLE_SID}.sql
                su $ORACLE_OWNER -c '${SQLDBA} <<EOED
connect / as sysdba;
@/tmp/endbkp${ORACLE_SID}.sql;
disconnect;
exit;
EOED'
            fi
            echo "Opening Oracle database $ORACLE_SID"
            su $ORACLE_OWNER -c '${SQLDBA} <<EOED
connect / as sysdba;
alter database open;
disconnect;
exit;
EOED'
        fi
    done
fi

if [ "$TASK" = "STOPALL" -o "$TASK" = "STOPDB" ]
then
    if [ ! -r $ORATAB ]
    then
        echo "Can't access $ORATAB"
        exit 1
    fi
    echo "\nORACLE databases shutdown\n*************************"
    DBRUN=""
    grep "^[a-z,A-Z,0-9]" $ORATAB | grep $DLNAME | ;
       awk -F":" '{if($3=="Y"||$3=="N")print $1,$2}' | ;
       while read ORACLE_SID ORACLE_HOME
    do
        PATH=$ORACLE_HOME/bin:$SAVE_PATH
        if [ -d $ORACLE_HOME/lib32 ]
        then
            LIBPATH=$ORACLE_HOME/lib:$ORACLE_HOME/lib32:$SAVE_LIBPATH
        else
            LIBPATH=$ORACLE_HOME/lib:$SAVE_LIBPATH
        fi
        LD_LIBRARY_PATH=$LIBPATH
        export ORACLE_HOME ORACLE_SID PATH LIBPATH LD_LIBRARY_PATH
        if [ -r $ORACLE_HOME/ops/ogms_profile.sh ]
        then
            . $ORACLE_HOME/ops/ogms_profile.sh
        fi
        export SQLDBA=""
        if [ -x $ORACLE_HOME/bin/svrmgrl ]
        then
            SQLDBA=svrmgrl
        elif [ -x $ORACLE_HOME/bin/sqlplus ]
        then
            SQLDBA="sqlplus /nolog"
        fi
        if [ "$SQLDBA" = "" ]
        then
            echo "Can't find or execute svrmgrl|sqlplus under $ORACLE_HOME/bin"
        elif oracle_proc_running $ORACLE_SID 1
        then
            [ -f /tmp/shut${ORACLE_SID}.flag ] && rm /tmp/shut${ORACLE_SID}.flag
            echo "Shutting immediately Oracle database $ORACLE_SID"
            echo "${SQLDBA} <<EOED\nconnect / as sysdba;\nshutdown immediate;\ndisconnect;\nexit;\nEOED\ntouch /tmp/shut${ORACLE_SID}.flag" >/tmp/shut${ORACLE_SID}.sh
            chmod 755 /tmp/shut${ORACLE_SID}.sh
            su $ORACLE_OWNER -c "/tmp/shut${ORACLE_SID}.sh >/tmp/shut${ORACLE_SID}.log 2>&1 &"
            DBRUN=${DBRUN}${ORACLE_SID}" "
        fi
    done

    # Wait for completion of the shutdowns
    while [ "$DBRUN" != "" -a $TIME_BEF_ABORT -gt 0 ]
    do
        sleep 15
        TIME_BEF_ABORT=`expr $TIME_BEF_ABORT - 15`
        DBRUNBIS=""
        for ORACLE_SID in $DBRUN
        do
            ORACLE_HOME=`grep "^${ORACLE_SID}:" $ORATAB | ;
                         awk 'BEGIN{FS=":"}{print $2}'`
            if [ -f /tmp/shut${ORACLE_SID}.flag ]
            then
                kill_oracle_process ${ORACLE_SID} &
                echo "Oracle database $ORACLE_SID has been shut properly"
            else
                DBRUNBIS=${DBRUNBIS}${ORACLE_SID}" "
            fi
        done
        DBRUN=$DBRUNBIS
    done

    # Abort any database still running
    for ORACLE_SID in $DBRUN
    do
        ORACLE_HOME=`grep "^${ORACLE_SID}:" $ORATAB | ;
                     awk 'BEGIN{FS=":"}{print $2}'`
        PATH=$ORACLE_HOME/bin:$SAVE_PATH
        LIBPATH=$ORACLE_HOME/lib:$SAVE_LIBPATH
        export ORACLE_HOME ORACLE_SID PATH LIBPATH
        if [ -r $ORACLE_HOME/ops/ogms_profile.sh ]
        then
            . $ORACLE_HOME/ops/ogms_profile.sh
        fi
        export SQLDBA=""
        if [ -x $ORACLE_HOME/bin/svrmgrl ]
        then
            SQLDBA=svrmgrl
        else
            SQLDBA="sqlplus /nolog"
        fi
        if oracle_proc_running $ORACLE_SID 1
        then
            echo "Aborting Oracle database $ORACLE_SID"
            su $ORACLE_OWNER -c '${SQLDBA} <<EOED
connect / as sysdba;
shutdown abort;
disconnect;
exit;
EOED'
            kill_oracle_process ${ORACLE_SID}
        fi
    done
wait
fi

if [ "$TASK" = "STOPALL" -o "$TASK" = "STOPLSNR" ]
then
    if [ ! -r $LSNRDESC ]
    then
        echo "Can't access $LSNRDESC"
        exit 1
    fi
    echo "\nORACLE listeners shutdown\n*************************"
    grep "^[a-z,A-Z,0-9]" $LSNRDESC | grep $DLNAME | ;
       awk -F":" '{if($3=="Y"||$3=="N")print $1,$2}' | ;
       while read LSNR_NAME ORACLE_HOME
    do
        if [ ! -x $ORACLE_HOME/bin/lsnrctl ]
        then
            echo "Can't find or execute $ORACLE_HOME/bin/lsnrctl"
        else
            PATH=$ORACLE_HOME/bin:$SAVE_PATH
            if [ -d $ORACLE_HOME/lib32 ]
            then
                LIBPATH=$ORACLE_HOME/lib:$ORACLE_HOME/lib32:$SAVE_LIBPATH
            else
                LIBPATH=$ORACLE_HOME/lib:$SAVE_LIBPATH
            fi
            LD_LIBRARY_PATH=$LIBPATH
            TNS_ADMIN=$ORACLE_HOME/network/admin
            export ORACLE_HOME PATH LIBPATH LD_LIBRARY_PATH TNS_ADMIN
            if [ -r $ORACLE_HOME/ops/ogms_profile.sh ]
            then
                . $ORACLE_HOME/ops/ogms_profile.sh
            fi
            if [ "$LSNR_NAME" = "DBSNMP" ]
            then
                su $ORACLE_OWNER -c "lsnrctl dbsnmp_stop"
            else
                su $ORACLE_OWNER -c "lsnrctl stop $LSNR_NAME"
            fi
        fi
    done
fi

date

exit 0

Use sed to repeat characters

echo an '=' sign then use the loop feature of sed to run round a loop a further 79 times replacing what you've got with the same thing plus and extra one!

echo "=" | sed -e :a -e 's/^=\{1,79\}$/&=/;ta'

Convert between lower and upper case using ansi standard

This way should be portable and give predictable results

LOWER_SID=$(echo "${SID}" | tr '[[:upper:]]' '[[:lower:]]')
UPPER_SID=$(echo "${SID}" | tr '[[:lower:]]' '[[:upper:]]')

Use sed to delete the first line of a file or command output printing from the second line onwards

Actually deceptively simple. Example remove the header from df output.

df -k | sed 1d

or more formally

df -k | sed -e '1,1d'

As a useful extension, suppose TO_DATA_DIR contains multiple directories, this will total the free space for all relevant filesystems
This one is specific to AIX, for other Unixes and Linux, use $4 instead of $3.

SPACE_AVAILABLE=$(df -k $TO_DATA_DIR | sed 1d | awk 'BEGIN {i=0} {i=i+$3} END {print i}')

Use sed to munge a controlfile into submission!

delete comment lines
delete all lines after the one starting with a semicolon
delete all blank lines
change reuse to set
change archivelog (with word boundaries) to noarchivelog
change old sid to new sid

sqlplus -s / as sysdba<<'EOSQL' >/dev/null
alter database backup controlfile to trace as '/tmp/regenerate_controlfile.sql' reuse resetlogs;
EOSQL
OLDSID='PROD'
NEWSID='TEST'
sed  -e '/^--/d' -e '/^\;/q' -e '/^ *$/d' -e 's/REUSE/SET/' -e 's/\\<ARCHIVELOG\\>/NOARCHIVELOG/' -e 's/\\"'${OLDSID}'\\"/\\"'${NEWSID}'\\"/' /tmp/regenerate_controlfile.sql

Find all files with lines containing a certain text

I am using this to search through all files that may contain a hard-coded password. On AIX, the -H switch to grep does not print out the filename. Using this technique, adding /dev/null to the search list, forces grep to print out the filename because it is now searching multiple files.

find / -name "*sh" -exec grep -n 'password_text_to_find' /dev/null {} + 2>/dev/null

Find all files where a certain text does not exist

Mostly we want to search for files containing a particular string but how to find those files that do not contain some text.
Using xargs instead of -exec forces the filename to be printed. -H should do this but doesn't seem to in this case.

find . -name "*cfg" | xargs grep -H -c TO_CV_HOST {} \; | grep ':0$' | cut -d: -f1

tar and gzip in one command using AIX

tar on AIX does not have the -z flag so it has to be done manually

cd $ORACLE_HOME/../..
tar -cvf - 19.12/ | gzip > /oracle/Patches/AIX.PPC64_19.12_gold.gz

and to unzip

cd $ORACLE_HOME/../..
gunzip < /oracle/Patches/AIX.PPC64_19.12_gold.gz | tar -xvf -

Use tar and gzip to copy files more efficiently across network between hosts

From the destination server, this will connect to the source, tar up each file and pull it back to the current server

ssh oracle@hn5211 "cd /oracle/ora_data4/iten3/ && tar -cf - . | gzip " | ( cd /oracle/ora_data2/iten/ && gunzip -c | tar -xvf - . )

double hash and double percent in shell variables to trim off characters from variables

# - trims from the start until the first occurrence of those characters matched by what follows the hash (abstemious)
## - trims from the start until the last occurrence of those characters matched by what follows the hash (greedy)
% - trims from the end backwards until the first occurrence of those characters matched by what follows the percent (abstemious)
%% - trims from the end backwards until the last occurrence of those characters matched by what follows the percent (greedy)

FILENAME="/home/bey9at77/hello.txt"
FILE_STUB1=${FILENAME##*\/}
FILE_STUB=${FILE_STUB1%\.*}
echo $FILE_STUB
hello
FILE_EXT=${FILENAME##*\.}
echo $FILE_EXT
txt

or
# - remove prefix reluctantly
## - remove prefix greedily
% - remove suffix reluctantly
%% - remove suffix greedily

words="do.re.mi"

echo ${words#*\.}
re.mi

echo ${words##*\.}
mi

echo ${words%\.*}
do.re

echo ${words%%\.*}
do

The best shell script ever. An example of how scripting should be done

It was written by someone at Oracle. Unfortunately (s)he did not put any author comment in it. Also unfortunately I cannot show it here as it is protected behind Oracles support website.
If you have an Oracle Metalink id, you can get the complete script here
Here is a snippet of a function that demonstrates proper commenting and a very good style.<br > The script is called physru.sh and upgrades an Oracle database in a rolling upgrade fashion by using a physical standby.
There are 4500 lines in the full script but it is so easy to read and understand because of the way it's written, it's like a breath of fresh air. Well done whoever you are!

        ###########################################################################
# NAME:        wait_mrp_active
#
# DESCRIPTION:
#   Wait for $MRP_START_TIMEOUT minutes to confirm that the MRP is active.  If
#   we can't detect an active MRP, abort the script with an error.
#
# INPUT(S):
#   Arguments:
#     $1: database user
#     $2: user password
#     $3: tns service name
#     $4: database unique name
#
#   Globals:
#     None
#
# RETURN:
#   None
#
        ###########################################################################
wait_mrp_active()
{
  display "confirming media recovery is running"
  l_wma_status=1
  l_wma_curtime=`perl -e 'print int(time)'`
  l_wma_exptime=`expr $MRP_START_TIMEOUT "*" 60`
  l_wma_maxtime=`expr $l_wma_curtime + $l_wma_exptime`
  while [[ "$l_wma_curtime" -lt "$l_wma_maxtime" ]]
  do
    is_mrp_running $1 $2 $3 $4
    if [[ "$?" -gt "0" ]]; then
      l_wma_status=0
      break
    fi

    sleep $MRP_START_INTERVAL
    l_wma_curtime=`perl -e 'print int(time)'`
  done
  chkerr $l_wma_status "could not detect an active MRP after $MRP_START_TIMEOUT minutes"
}

Use Input Field Separator (IFS) to split a set of numbers into its constituent parts

This example splits the MacOS version into major and minor numbers

OLDIFS=$IFS
IFS='.' read osvers_major osvers_minor osvers_dot_version <<< "$(/usr/bin/sw_vers -productVersion)"
IFS=$OLDIFS

Split a file into pieces

This will split a file into separate files with 20000 lines in each (except maybe the last one!)

awk 'NR%20000==1{x="alert_lbk5t.log."++i;}{print > x}' alert_lbk5t.log

Strip out duplicate lines with this clever awk trick

Assigns each line to an associative array element and prints the line if it does not already exist in the array

awk '!visited[$0]++' your_file > deduplicated_file

split string (eg. filename) into separate variables using set

FILENAME="split_this_into_bits.txt"
set $(echo "${FILENAME}" | sed 's/_/ /g')
echo $4 $3 $2 $1 $5
bits.txt into this split

Centre align text on a line in bash shell

Doesn't work in Korn shell due to %*s

#COLUMNS=$(tput cols)   # width of the current window
COLUMNS=80
title="Hello world!"
printf "%*s\n" $(((${#title}+$COLUMNS)/2)) "$title"

and as a one-liner

printf "%*s\n" $(( ( $(echo $* | wc -c ) + 80 ) / 2 )) "$*"

This works in Korn shell…

TITLE="$1"
LINEWIDTH=80

LPAD=$(((${#TITLE}+$LINEWIDTH)/2))
printf %${LPAD}s "$TITLE"

Right justify text (or repeat characters) on a line with leader dots!

function rpadwait {
    text=$1
# -------------------------
# suppress newline for echo
# -------------------------
N=
C=
if echo "\c" | grep c >/dev/null 2>&1; then
    N='-n'
else
    C='\c'
fi
    echo ${N} "${text}${C}" | sed -e :a -e 's/^.\{1,80\}$/&\./;ta'
}

cron jobs not submitted - daemon run out of child processes?

Sometimes have problems with NFS mounts and this causes cron jobs to hang. If they are scheduled to run regularly, eventually cron will no longer be able to start any more jobs.

Check the cron log /var/adm/cron/log to see if there are any errors or other messages around the time the jobs should run.

If cron has hit its process limit (default 100), it will try again after a number of seconds (default 60).
Both the number of jobs and wait time are configured in the file /var/adm/cron/queuedefs. If it is unusual for cron to be running so many jobs, you can check the process table to view the jobs cron has created. These jobs will have parent process id (PPID) of the cron daemon.

$ ps -ef | grep cron | grep -v grep
    root  6750314        1   0   Apr 24    # 3:39 /usr/sbin/cron
solax025:root[/home/root]# ps -T 6750314
      PID    TTY  TIME CMD
  6750314    # 3:39 cron
 21168296    # 0:00     \--bsh
 58982414    # 0:00         \--sadc

In this example, we only have 1 job running

Find long-running processes with a cron job

Processes running longer than 24 hours have a date instead of a start time…

58 08,14 * * * /home/ibmtools/scripts/oracle/dosh -vc "ps -ef|grep 'ibmtools/scripts/oracle'|perl -nae 'print if \$F[4] !~/:/'" >/tmp/lrp.txt; [[ $(grep -c ibmtools /tmp/lrp.txt) -ne 0 ]] && cat /tmp/lrp.txt|mailx -s '*** long running processes - please check ***' [email protected]

Process command line arguments in shell

Borrowed from Mozilla Firefox installer

# Command line arg defaults
#
moz_debug=0
moz_debugger=""
moz_debugger_args=""
#
#
# Parse the command line
#
while [[ $# -gt 0 ]]
do
  case $1 in
    -g | --debug)
      moz_debug=1
      shift
      ;;
    -d | --debugger)
      moz_debugger=$2;
      if [[ "${moz_debugger}" != "" ]]; then
        shift 2
      else
        echo "-d requires an argument"
        exit 1
      fi
      ;;
    -a | --debugger-args)
      moz_debugger_args=$2;
      if [[ "${moz_debugger_args}" != "" ]]; then
        shift 2
      else
        echo "-a requires an argument"
        exit 1
      fi
      ;;
    *)
      break;
      ;;
  esac
done
#
#
# Program name given in $1
#
if [[ $# -gt 0 ]]; then
    MOZ_PROGRAM=$1
    shift
fi
#
# Program not given, try to guess a default
#
if [[ -z "$MOZ_PROGRAM" ]]; then
    ##
    ## Try this script's name with '-bin' appended
    ##
    if [[ -x "$MOZ_DEFAULT_NAME" ]]; then
        MOZ_PROGRAM=$MOZ_DEFAULT_NAME
        ##
        ## Try mozilla-bin
        ##
    elif [[ -x "$MOZ_APPRUNNER_NAME" ]]; then
        MOZ_PROGRAM=$MOZ_APPRUNNER_NAME
    fi
fi
#
#
#
# Make sure the program is executable
#
if [[ ! -x "$MOZ_PROGRAM" ]]; then
    moz_bail "Cannot execute $MOZ_PROGRAM."
fi

Carry XWindows settings across sessions

# ----------------------
# push XWindows settings
# ----------------------
[[ "$(uname)" == "SunOS" ]] && PATH=/usr/openwin/bin:$PATH
WHOAMI=$(id | awk -F')' '{print $1}' | awk -F'(' '{print $2}')
xauth list > /tmp/xauth_list_$WHOAMI
chmod 777 /tmp/xauth_list_$WHOAMI

echo $DISPLAY > /tmp/xdisplay_$WHOAMI
chmod 777 /tmp/xdisplay_$WHOAMI

Cross-platform version of whoami

WHOAMI=$(id | awk -F')' '{print $1}' | awk -F'(' '{print $2}')

Set terminal title from command line

Put something like this in the .profile
-n do not output the trailing newline
-e enable interpretation of backslash escapes

echo -en "\033]0;`hostname`\007"

echo -en "\033]2;`hostname`\007"

Remove blank lines and comments (also indented ones) from a file

awk -F\: '!/^($|[:space:]*#)/ {print $2}' /etc/oratab | sort | uniq

or as part of a script that removes comments and blank lines from all Korn shell scripts in a directory

#!/usr/bin/ksh
for i in *ksh; do
    perl -p -i -e 's/^\s*#[^!]*$//; s/^\s*$//' $i
done

Return elements of an array in Korn shell

From here
Could be used to separate the columns of an SQL select when returning to the shell
This approach eliminates the need to put quotes around text with spaces in it.

echo $KSH_VERSION
x="Red,Yellow is a color,Blue"
oIFS=$IFS
IFS=,
y=($x)
IFS=$oIFS
echo ${y[1]}

A decent Unix Prompt

export PS1="`uname -n`:`whoami`[\${PWD}]# "
or
export PS1='($?) $'ORACLE_SID" "`whoami`"@"`uname -n`":"'$'PWD"> "
export EDITOR=vi

Simple arithmetic

pipe the calculation into the shell calculator

space_in_kb=$(echo $1 / 1024 | bc)

Calculate the remainder (modulo) of a division calculation

if [[ $(echo "${NUMBER} % 2" | bc) -eq 0 ]]; then
    echo "${NUMBER} is even"
else
    echo "${NUMBER} is odd"
fi

or do it in awk if scientific notation maybe involved

function calc { awk "BEGIN{print $*}"; }

if [[ $(calc "${SPACE_USED} + ${SPACE_AVAILABLE} - ${DATABASE_SIZE") -le 0 ]]; then
    echo "NOK"
fi

Script encryption and passphrase protection

Encrypt a shell script with the ability to execute the encrypted version

from commandlinefu.com

scrypt(){ [ -n "$1" ]&&{ echo '. <(echo "$(tail -n+2 $0|base64 -d|mcrypt -dq)"); exit;'>$1.scrypt;cat $1|mcrypt|base64 >>$1.scrypt;chmod +x $1.scrypt;};}

Virtual host configuration in Apache http.conf

<VirtualHost *:80>
        ServerName dbamon
        DocumentRoot "/Volumes/data/Sites/dbamon_push"
        <Directory "/Volumes/data/Sites/dbamon_push">
                Options Includes FollowSymLinks
                AllowOverride All
                Order allow,deny
                Allow from all
        </Directory>
</VirtualHost>

and in /etc/hosts, add…

127.0.0.1  dbamon

Mount a website (or any other remote resource) locally using WebDav

Redhat/CentOS

yum install fuse-davfs2
or
yum install wdfs.x86_64

Debian

apt-get install davfs2

then…

sudo mkdir /mnt/webdav  # or whatever you'd like to call the directory
sudo mount.davfs [-o option[,option]...] device mount_point

In man's terms, that last line would translate to:

id  #suppose this returns uid=501 and gid=502
sudo mount.davfs -o 501,502 https://your/web/site /mnt/webdav

Mac OSX

osascript -e ' mount volume "http://username:[email protected]:portnum" '
or
osascript -e ' mount volume "http://[email protected]:portnum" '
or
osascript -e ' try mount volume "http://webdav.address:portnum" '
or
mount -t webdav http://webdav.address:portnum /mnt/webdav  # this one won't show up in the Finder Sidebar.

Using Borg to backup to a remote server via sshfs

BorgBackup is a deduplicating backup program. Optionally supporting compression and authenticated encryption

Use sshfs to mount a remote filesystem locally

[https://vimeo.com/54505525?cjevent=331dfbdafd4f11e880e4005e0a180514|The Black Magic Of SSH / SSH Can Do That?]]
Fedora 26: Install sshfs for SSH client

Presuming Fedora 26+. Install sshfs

sudo dnf -y install sshfs

Mount the filesystem locally

mkdir borg
sshfs [email protected]:borg borg

read: Connection reset by peer

Debug the reason it does not mount the filesystem

[dbahawk@fedora ~]$ sshfs -odebug,sshfs_debug,loglevel=debug -o Compression=no -o allow_root -o transform_symlinks [email protected]:borg borg
SSHFS version 2.10
FUSE library version: 2.9.7
nullpath_ok: 0
nopath: 0
utime_omit_ok: 0
executing <ssh> <-x> <-a> <-oClearAllForwardings=yes> <-ologlevel=debug> <-oCompression=no> <-2> <[email protected]> <-s> <sftp>
read: Connection reset by peer
[dbahawk@fedora ~]$ sshfs -odebug,sshfs_debug,loglevel=debug -o Compression=no -o allow_root -o transform_symlinks [email protected]:borg borg
SSHFS version 2.10
FUSE library version: 2.9.7
nullpath_ok: 0
nopath: 0
utime_omit_ok: 0
executing <ssh> <-x> <-a> <-oClearAllForwardings=yes> <-ologlevel=debug> <-oCompression=no> <-2> <[email protected]> <-s> <sftp>
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug1: /etc/ssh/ssh_config.d/05-redhat.conf line 8: Applying options for *
debug1: Connecting to 192.168.1.11 [192.168.1.11] port 22.
debug1: Connection established.
debug1: key_load_public: No such file or directory
debug1: identity file /home/dbahawk/.ssh/id_rsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/dbahawk/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/dbahawk/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/dbahawk/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/dbahawk/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/dbahawk/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/dbahawk/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/dbahawk/.ssh/id_ed25519-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/dbahawk/.ssh/id_xmss type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/dbahawk/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_7.7
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4
debug1: match: OpenSSH_7.4 pat OpenSSH* compat 0x04000000
debug1: Authenticating to 192.168.1.11:22 as 'stuart'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: [email protected]
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: none
debug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: none
debug1: kex: [email protected] need=32 dh_need=32
debug1: kex: [email protected] need=32 dh_need=32
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:+h02thqjO0/SoVPccBz1RTudBCJ+V7g6MKu4E4CCToQ
debug1: Host '192.168.1.11' is known and matches the ECDSA host key.
debug1: Found key in /home/dbahawk/.ssh/known_hosts:1
debug1: rekey after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey after 4294967296 blocks
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Trying private key: /home/dbahawk/.ssh/id_rsa
debug1: Trying private key: /home/dbahawk/.ssh/id_dsa
debug1: Trying private key: /home/dbahawk/.ssh/id_ecdsa
debug1: Trying private key: /home/dbahawk/.ssh/id_ed25519
debug1: Trying private key: /home/dbahawk/.ssh/id_xmss
debug1: Next authentication method: password
[email protected]'s password: 
debug1: Authentication succeeded (password).
Authenticated to 192.168.1.11 ([192.168.1.11]:22).
debug1: channel 0: new [client-session]
debug1: Requesting [email protected]
debug1: Entering interactive session.
debug1: pledge: network
debug1: client_input_global_request: rtype [email protected] want_reply 0
debug1: Sending environment.
debug1: Sending env LANG = en_GB.UTF-8
debug1: Sending subsystem: sftp
subsystem request failed on channel 0
read: Connection reset by peer

The last 3 lines tell us that sftp server is probably not enabled on the remote host. Verify by trying sftp manually…

[dbahawk@fedora ~]$ sftp [email protected]
[email protected]'s password: 
subsystem request failed on channel 0
Connection closed
[dbahawk@fedora ~]$ sftp -P10022 [email protected]
[email protected]'s password: 
Connected to [email protected].
sftp> exit

Aah, I forgot it was installed but listening on a different port.

$ sshfs -o port=10022 [email protected]:borg borg
$

Success.

Add current hostname to list of hosts on an xcat server

CMD="nodels"
HOST=`hostname`
(echo "$HOST"; $CMD) | while read server
do
    echo "server:$server"
done

What is todays 'Day Of the Year' number?

DOY=`perl -e 'print sub{$_[7]}->(localtime)+1;'`

Convert Julian day numbers to dates

for day in 8 33 36 61 64 91 96 121 126 152 155 182 187 215 218 244 247 274 279 306 309 335 338 365; do date -d "`date +%Y`-01-01 +$(( ${day} - 1 ))days" +%d-%m-%Y; done

Send crontab job output to a date formatted log file

This will run a job every 5 minutes and send the output to a file ending with a time in hours and minutes.
The thing to note is the escaped percent signs. This is because a % sign is interpreted by cron to mean a newline character. Everything after the first % is treated as input to the program.

*/5 *  * * * /var/www/cgi-bin/dbamon_collector.ksh       >/tmp/dbamon_collector.log.$(date "+\%H\%M") 2>&1

Edit crontab file without crontab -e

It can happen that you need to add or modify a line in the crontab of many users or across many servers at once.
In principle, there's nothing wrong with modifying the crontab file directly. You just lose the advantages of file locking (and syntax checking) that crontab -e offers.
Here we take a backup of the current crontab, print it out, echo an extra command and ask cron to use these as input (thus overwriting the existing crontab file). Just don't run this close to midnight

crontab -l > /tmp/crontab.`date '+%Y%m%d'`
(
cat /tmp/crontab.`date +'%Y%m%d'`
echo "02 10 * * * /home/ibmtools/scripts/oracle/export_parfile.ksh -s SID -f JDBEOP1.parfile"
) | crontab -

or

crontab -l > /tmp/crontab.backup
crontab -l > /tmp/crontab.$$
perl -p -i -e 's!backup_send_tsm_dump!backup_export2tsm!g' /tmp/crontab.$$
crontab /tmp/crontab.$$
rm /tmp/crontab.$$

or

crontab -l >$HOME/crontab.$(date '+%Y%m%d')
crontab -l | perl -p -e 's|/nas/software/oracle/scripts|/oracle/scripts|' | crontab

Use shell to convert a number in scientific notation to normal

var2convert='1.2345678e3'
printf -v var2convert "%.f" $var2convert
echo $var2convert  # magic!

Check for jobs running longer that 24 hours

Run from the management server across all Unix servers. Checks the 5th column in a ps listing. If it doesn't find a : (time separator), the process is running longer than 24 hours.

# ---------------------------------------
# report on long running oraibm processes
# ---------------------------------------
# ---------------------------------------
# report on long running oraibm processes
# ---------------------------------------
01 17,14 * * * rm -f /tmp/lrp.txt;/home/ibmtools/scripts/oracle/dosh -vc \"ps -ef|egrep 'oraibm|scripts/oracle'>/tmp/lrp.txt;perl -nae 'print if \$F[4] !~/:/' /tmp/lrp.txt\" >>/tmp/lrp.txt;[[ $(egrep -c 'oraibm|scripts/or
acle' /tmp/lrp.txt) -ne 0 ]] && cat /tmp/lrp.txt|mailx -s '*** long running processes - please check ***' [email protected]

Global search and replace with perl (restricted to list of files provided by Unix find command)

find . -type f -exec perl -i -pe 's/something/else/g' {} \;

perl function to make filenames lower case

function lower {
   perl -e 'for (@ARGV) { rename $_, lc($_) unless -e lc($_); }' *
}

From the management server, search the TNS listener port for each database on a server and make an inline substitution in the ITM config files!

for i in `/home/ibmtools/scripts/oracle/dosh -c "ls -al /opt/IBM/ITM/config/*rz*cfg|grep -v lrwx"|awk '{print $NF}'`; do
    server=`echo $i|cut -d_ -f1 | awk -F'/' '{print $NF}'`
    db=`echo $i|cut -d'.' -f1 | awk -F'_' '{print $NF}'`
    OH=`ssh $server grep "^$db" /etc/oratab|cut -d: -f2`
    LISTENERPORT=`ssh $server cat $OH/network/admin/listener.ora|perl -00 -ne 'print $1 if /'$db'.*PORT.*=.*(\d{4})/s'`
    ssh $server perl -p -i.bak -e 's/1521/'$LISTENERPORT'/' $i
    ssh $server ls -al ${i}*
done

Run a job from cron every Nth day of the month

Example. Execute a job every third Saturday of the month.
Paste this into a file called calenday and put it in /usr/local/bin so it's (probably) on the PATH

#!/usr/bin/ksh

# ix is the week number of the month ("2"nd Friday of the month, "3"rd Tuesday of the month)
# dy is the day number in Unix format (0 for Sunday, 1 for Monday, ... 6 for Saturday)
# eg. "calenday 3 6" returns the date of 3rd Saturday of the month.

ix=$1
dy=$2
SCHAR=$((($dy*2)+$dy+1))
ECHAR=$(($SCHAR+1))
cal `date '+%m %Y'` | egrep "\<[0-9]{1,2}\>" | cut -c${SCHAR}-${ECHAR} | xargs | awk {'print $'$ix'}'

Now in crontab, you should be able to do something like this:

15 20 * * * [[ `calenday 3 6` -eq `/bin/date '+%d'` ]] && su - oracle -c "run_my_backup.ksh"

This will also work on some Unices..
We send backups to a special TSM node on the second Friday of each month. This report must run a day later - but that is not guaranteed to be the second Saturday or even the 3rd. So…

30 12 8-14 * 5 sleep 86400 && su - oracle -c "/usr/bin/perl -ne 'print if /ORX_M_SOL/ .. /^STOP/' /home/oracle/incoming/dbamon_spool_tsm_*.SOL | grep Archive | grep -v Client | mailx -s 'Monthly TSM backups' [email protected]"

ps listing does not show start time after 24 hours

But you can see elapsed time using your own ps command

/usr/bin/ps -eo etime,user,pid,ppid,cpu,start,tty,time,args|tail -n +2|sort

ps -ef cuts off args cmd column on Solaris

To see more than 80 characters of the last column on Solaris

This shows all the individual arguments to the command

pargs <pid>

This shows the ps listing in 'compatibility' mode (there are more compatibility commands in /usr/xpg4/bin)

/usr/ucb/ps auww

To see more than 80 characters of the last column on AIX

This shows the full argument listing of a process (NOTE: no minus sign!)

ps eww <pid>

Remove blank / empty lines from vi

Maybe you cut and pasted a file from Windows and it's full of blank lines and Control-M's now
There are several methods but I think this is the easiest to remember

:g/^$/d

Right pad a variable

a function like rpad in SQL but for Shell

function rpad {
text=$1
padwidth=$2
padchar=$3
echo "$text" | sed -e :a -e 's/^.\{1,'$padwidth'\}$/&\'$padchar'/;ta'
}

Connect to a Windows server from Linux using rdesktop

My remmina stopped working so rolled my own. Very simple really. Put this is a shell.

tsocks rdesktop -z -P -x m -a 16 -d MAIND -u sbarkley -p ****** -r disk:SOL=$HOME/Documents/SOL -g 95% 150.251.112.38 &

where...
-z  #enables compression
-P  #enables bitmap caching (saves network traffic)
-x m#disables eye-candy features
-a 16 - reduce colour pallete to 16 colours
-d  #domain to connect to
-u  #username
-p  #password
-r  #setup a shared folder
-g  #geometry (use W x H or percentage)

Slight problems with rdesktop not working 100% of the time. Now using xfreerdp. Seems better…

xfreerdp -g 90% --ignore-certificate --gdi sw -K -d wdcrhbp05 -u oraibm -p "`cat $HOME/scripts/.oraibm.password`" -T "wdcrhbp05_oraibm" --plugin cliprdr --plugin rdpdr --data disk:SOL:/home/bey9at77/Documents/SOL -- --plugin rdpsnd --data alsa -- 150.251.112.25 &

Reset your password bypassing password rules

must be done as root

echo "user:new_password" | chpasswd

Sum the sizes of all files of an ls listing

It'll check to see if the sum of filesizes corresponds with the df -g (or h) listing (can get messed up due to open but deleted files)

cd /oracle/export
df -g .
find . -name "*dmp*" -ls |  awk '{ SUM += $7} END { print SUM/1024/1024/1024 }'

Mount an iso image under Linux

mkdir -p /mnt/cdrom
mount -o loop /dev/cdrom /mnt/cdrom

How many processors on the machine?

AIX

lsdev -C|grep Process|wc -l

Solaris

psrinfo -v|grep "Status of processor"|wc -l

Linux

cat /proc/cpuinfo|grep processor|wc -l

Quick, simple, understandable example of how to use RRDTool

RRDTool

Use expect to respond automatically to interactive programs

#!/usr/bin/expect -f
spawn /usr/tivoli/tsm/client/oracle/bin64/tdpoconf passw -tdpo_optfile=/oracle/[lindex $argv 0]/admin/tdpo.opt
expect "assword:" {send "password\r"}
expect "assword:" {send "password\r"}
expect "assword:" {send "password\r"}

Use expect to allow file copy with scp (if ssh keys are not an option)

#!/usr/bin/expect -f
spawn scp /home/oracle/.profile oracle@hn512:/tmp/prfl
set pass "thepassword"
expect {
        password: {send "$pass\r" ; exp_continue}
        eof exit
}

Send logging of a shell script to file from inside the script

Spotted this seemingly simple method in $FND_TOP/bin/batchmgr.sh

# Save output
nohup sh << end_logging >> $logfile 2>&1 &
...
script contents
...
end_logging

Log output of a shell script to both terminal and a file using a named pipes

or “How to send shell output to screen/stdout as well as to a logfile using tee and redirection with exec”

examples here on unix.com
general info on redirections here at catonmat.net

Starting with this example found in Oracle script $ORACLE_HOME/root.sh

#
# Silent variable is set based on :
# if OUI_SILENT is true or if SILENT_F is 1
#
  if [ "${OUI_SILENT}" = "true" -o "$SILENT_F" ]
  then
    SILENT=1
  else
    SILENT=0
  fi

  if [ "${OUI_STDOUT}" = "true" -o "$STDOUT_F" ]
  then
    STDOUT=1
  else
    STDOUT=0
  fi

  if [ $SILENT -eq 1 -a $STDOUT -eq 0 ]
  then
    $ECHO "Check $LOG for the output of root script"
    exec > $LOG 2>&1
  else
    mkfifo $LOG.pipe
    tee < $LOG.pipe $LOG &
    exec > $LOG.pipe 2>&1
    rm $LOG.pipe
  fi

Using a demonstration as an example of how exec works

#!/bin/ksh

LOGFILE="$0.log"
PIPE="$0.pipe"
[[ -e "$PIPE" ]] || mkfifo "$PIPE"    # or mknod -p ??

# make a new channel(3) and copy channel 1's destination as its own (does NOT POINT TO channel 1's destination)
# this allows the normal output to continue to STDOUT but also get printed to whatever file is attached to channel 3
exec 3>&1

# anything coming in on the pipe, send it to $LOGFILE and to channel 3
tee "$LOGFILE" <"$PIPE" >&3 &

# redirect STDOUT to the pipe
exec > "$PIPE"

echo "going to default output"
echo "forcing to channel 1" >&1
echo "forcing to channel 2" >&2
echo "forcing to channel 3" >&3

rm -f "$PIPE"

Another example of how to set up simultaneous output to both terminal and a logfile using file descriptors

exec 3<> /tmp/foo  #open fd 3 for r/w
echo "test" >&3
exec 3>&- #close fd 3.

exec 3<> myfile.txt
while read line <&3
do {
  echo "$line"
  (( Lines++ ));                   #  Incremented values of this variable
                                   #+ accessible outside loop.
                                   #  No subshell, no problem.
}
done
exec 3>&-

echo "Number of lines read = $Lines"     # 8

Mmm. See our output and also tee it to a log file!

#!/bin/bash

echo hello

if test -t 1; then
    # Stdout is a terminal.
    exec >log
else
    # Stdout is not a terminal.
    npipe=/tmp/$$.tmp
    trap "rm -f $npipe" EXIT
    mknod $npipe p
    tee <$npipe log &
    exec 1>&-
    exec 1>$npipe
fi

echo goodbye

and another example

Found at https://www.unix.com/shell-programming-and-scripting/70726-how-redirect-script-output-inside-script.html

#!/bin/bash
#Objective : To redirect the stdout & stderr to two different files from within the script and without sacrificing the scripts output

mkfifo /tmp/out.pipe /tmp/err.pipe

exec 3>&1 4>&1

tee /tmp/std.out < /tmp/out.pipe >&3 &
pid_out=$!
exec  1>/tmp/out.pipe

tee /tmp/std.err < /tmp/err.pipe >&4 &
pid_err=$!
exec  2>/tmp/err.pipe

log() {
 echo "`date` : $@ "
}

log "Starting the process"
log "Listing directories now..."
ls -l
ls -l Log_to_Stderr
exec 1>&3 3>&- 2>&4 4>&-
wait $pid_out
wait $pid_err
rm /tmp/out.pipe /tmp/err.pipe
log "End of Processing"

More elaborate example

found here on stackoverflow

#!/bin/sh

# Author: Harvey Chapman <hchapman _AT_ 3gfp.com>
# Description: POSIX shell functions that can be used with tee to simultaneously put
#              stderr and stdout to both a file and stdout
#
# Based on:
#    Re: How to redirect stderr and stdout to a file plus display at the same time
#    http://www.travishartwell.net/blog/2006/08/19_2220

#
# Original example function from Travis Hartwell's blog.
# Note: I've made minor changes to it.
example()
{
  OUTPUT_LOG=output.log
  OUTPUT_PIPE=output.pipe

  # This should really be -p to test that it's a pipe.
  if [[ ! -e $OUTPUT_PIPE ]]; then
      mkfifo $OUTPUT_PIPE
  fi

  # This should really be -f to test that it's a regular file.
  if [[ -e $OUTPUT_LOG ]]; then
      rm $OUTPUT_LOG
  fi

  exec 3>&1 4>&2
  tee $OUTPUT_LOG < $OUTPUT_PIPE >&3 &
  tpid=$!
  exec > $OUTPUT_PIPE 2>&1

  echo "This is on standard out"
  echo "This is on standard err" >&2

  exec 1>&3 3>&- 2>&4 4>&-
  wait $tpid

  rm $OUTPUT_PIPE
}

# A slightly reduced version of example()
example2()
{
  OUTPUT_LOG=output.log
  OUTPUT_PIPE=output.pipe

  rm -f $OUTPUT_PIPE
  mkfifo $OUTPUT_PIPE
  rm -f $OUTPUT_LOG

  tee $OUTPUT_LOG < $OUTPUT_PIPE &
  tpid=$!

  exec 3>&1 4>&2 >$OUTPUT_PIPE 2>&1

  echo "This is on standard out"
  echo "This is on standard err" >&2

  exec 1>&3 3>&- 2>&4 4>&-
  wait $tpid
  rm -f $OUTPUT_PIPE
}

#
# Logging methods based on above. See the example below for how to use them.
#

# Usage: start_logging [[delete_existing_logfile]]
start_logging()
{
  # Check to see if OUTPUT_LOG and OUTPUT_PIPE need to be defined.
  if [[ -z "$OUTPUT_LOG" ]]; then
    OUTPUT_LOG=output.log
  fi
  if [[ -z "$OUTPUT_PIPE" ]]; then
    OUTPUT_PIPE=output.pipe
  fi
  # Make sure that we're not already logging.
  if [[ -n "$OUTPUT_PID" ]]; then
    echo "Logging already started!"
    return 1
  fi

  # Always remove the log and pipe first.
  rm -f $OUTPUT_PIPE
  # Delete the logfile first if told to.
  if [[ "$1" = delete_existing_logfile ]]; then
    rm -f $OUTPUT_LOG
  fi

  mkfifo $OUTPUT_PIPE
  tee -a $OUTPUT_LOG < $OUTPUT_PIPE &
  OUTPUT_PID=$!

  exec 3>&1 4>&2 >$OUTPUT_PIPE 2>&1
}

stop_logging()
{
  # Make sure that we're currently logging.
  if [[ -z "$OUTPUT_PID" ]]; then
    echo "Logging not yet started!"
    return 1
  fi
  exec 1>&3 3>&- 2>&4 4>&-
  wait $OUTPUT_PID
  rm -f $OUTPUT_PIPE
  unset OUTPUT_PID
}

example3()
{
  start_logging
  #start_logging delete_existing_logfile
  echo "This is on standard out"
  echo "This is on standard err" >&2
  stop_logging
}

Execute an SQL file from a shell and email the results to a user

This is bare-bones with no error-checking but works as tested.
The idea is to gather up the results throughout the day (however many times it is run from crontab) and the first time it is run on the following day, it will zip up the file from the previous day and mail to to a user.

#!/usr/bin/ksh
export PATH="/usr/local/bin:${PATH}"
export ORACLE_SID="$1"
ORAENV_ASK=NO
. oraenv
SPOOLDIR="/tmp"
SPOOLSTUB="${ORACLE_SID}_pga_stats"
SPOOLFILE="${SPOOLSTUB}_$(date '+%y%m%d').csv"
sqlplus '/ as sysdba'<<EOSQL
@/oracle/scripts/pga_mem_stats '${SPOOLDIR}/${SPOOLFILE}'
EOSQL
FILE2SEND=$(ls -1tr "${SPOOLDIR}/${SPOOLSTUB}"*|grep -v "${SPOOLFILE}")
[[ "${FILE2SEND}" == "" ]] && echo "No file ready to email yet" && exit 0
gzip "${FILE2SEND}"
(echo "username,sid_serial,ospid,logon_time,time_now,program,machine,port,status,cpu_secs,pga_max_mem,pga_alloc_mem,pga_used_mem,pga_freeable_mem"; uuencode "${FILE2SEND}.gz" "${FILE2SEND}.gz") | ;
mailx -s "PGA stats file for ${ORACLE_SID} (${FILE2SEND})" -r "$(whoami)@$(hostname)" "[email protected]"
echo mv "${FILE2SEND}.gz" "${SPOOLDIR}/emailed_${SPOOLFILE}.gz"
mv "${FILE2SEND}.gz" "${SPOOLDIR}/emailed_${SPOOLFILE}.gz"

RedHat root filesystem has gone read-only

How to fix read only root file system

Kill all processes for a user

for prc in `ps -ef | grep -E "^ +[o]racle" | awk '{print $2}'`; do
    kill $prc
done

isRGHere

Checks if resource group is on this leg of an HACMP cluster. Returns 0 if true else 1.

#!/usr/bin/ksh

SCRIPT=`basename $0`

function rctest
{
exitcode=$1
msg=$2
if [[ $exitcode -ne 0 ]]; then
        echo "********************************************************************************"
        echo ";
Script $SCRIPT finished with errors."
        echo "$msg."
        echo "Returncode : $exitcode."
        echo ";
          * ***************************************************************************"

        fi
exit $exitcode

}

RGINFO=/usr/es/sbin/cluster/utilities/clRGinfo
[[ ! -f $RGINFO ]] &&  rctest 1 "clRGinfo not found"

if [[ $# -eq 1 ]]
then
        RG=`echo $1 | cut -c 1-14`
else
        rctest 10  "Usage: `basename $0` <RG name>"
fi

$RGINFO |grep -qwp $RG ||  rctest 9 "$RG is not defined"

THISNODE=`/usr/es/sbin/cluster/utilities/get_local_nodename | cut -c 1-14`
$RGINFO |grep -wp $RG |grep -w $THISNODE |grep -wq ONLINE

or this one-liner will work by returning ONLINE or OFFLINE or nothing in case of non-clustered machine

/usr/es/sbin/cluster/utilities/clRGinfo|grep $(hostname|awk -F. '{print $1}')|awk '{print $(NF-1)}'

Find cluster name for current node of an hacmp cluster

/usr/es/sbin/cluster/utilities/cltopinfo|grep 'Cluster Name'|awk '{print $NF}'

AIX: add a user to a group

chgrpmem -m + oracle ibmtls

Find swear words in scripts by comparing it to a list found on the internet

wget "http://www.bannedwordlist.com/lists/swearWords.csv" -O /tmp/s ; for word in $(cat /tmp/s | sed -e 's/ /_/g' -e 's/,/ /g') ; do grep -wR $word *; done | less

grep -p (for paragraph) works on AIX but not on Linux or Solaris

Use awk instead

awk 'BEGIN {FS="\n" RS="\n\n"} /search pattern/ { do something }' <file>
/usr/xpg4/bin/awk 'BEGIN {RS="\n\n";FS="\n"} /AGRHDWQ1/ {print $2}' dsm.sys | awk '{print $NF}'

this prints the last word of the second line in the paragraph in dsm.sys that contains the search term AGRHDWQ1.
Or slightly simpler…

awk -v RS='' '/NHAPPLP1/' /etc/tsm/dsm.sys     # (use /usr/xpg4/bin/awk on Solaris)

or, case insensitively:

awk -v RS='' 'tolower($0) ~/so_u_clubqa_orx_d_cab/' /etc/tsm/dsm.sys

Using -v means you don't have to use a BEGIN section.

debug/redirect log of a shell

#!/usr/bin/ksh
exec 2>/tmp/mytest

Different ways of Iteration in korn shell with a while loop

IFS="|"
exec 0<$statfile
while read host db started stopped
do
    rrdfile="export_duration_${host}_${db}.rrd"
    $RRDTOOL update ${RRDDIR}/${rrdfile} $started:$started $stopped
done

or

while read host db started stopped
do
    rrdfile="export_duration_${host}_${db}.rrd"
    $RRDTOOL update ${RRDDIR}/${rrdfile} $started:$started $stopped
done <statfile

or

    cat $statfile | sort -nk4 | while IFS=\| read host db type started stopped
    do
        [[ "$stopped" == "" ]] && continue
        rrdfile="export_duration_${host}_${db}.rrd"
        $RRDTOOL update ${RRDDIR}/${rrdfile} ${started}:${started}:${stopped}
        [[ $? -ne 0 ]] && echo "nok: $?"
    done

Is it a directory or a filesystem mountpoint?

for i in /cln/exp/ora_data3/*; do
    FS=""
    MNT="$(df -g $i|grep -v Filesystem|awk '{print $NF}')"
    if [[ "$i" == "${MNT}" ]]; then
        FS=" (FS)"
    fi
    echo "${i}${FS}"
done

Filesystem 100% full, what's taking up all the space?

find /oracle/endur -xdev -ls|sort -nr +6|head

or

/dev/esb01fs010001      1.00      0.00  100%     1838    69% /oracle
cd /oracle
du -gsx * | sort -n
0.00    checkpoints
0.00    flash_recovery_area
0.00    lost+found
0.00    oraInst.loc
0.00    oraInventory
0.09    admin
0.99    diag

cd diag and repeat until culprits are found

df -g shows filesystem full but du -gsx shows it as not full!

Probably caused by some large files being deleted while a process was still writing to them (and is continuing to write to them).

Solution is either kill the process that was writing to the file (descriptor) or find the file descriptors associated with the removed files and close them.

This lists all the files under that filesystem that have been deleted (unlinked)

lsof +aL1 /cln/tst sort -nk 7 | tail -10

Then look through the /proc filesystem for the file descriptors linked to the removed files

ls -ltr /proc/<pid>/fd | grep <piece of the filename>

Then we need to use a debugger to close the descriptors

dbg -p <pid>

call close (<file descriptor>)

quit

Capitalise the first letter of a word in Korn Shell

Not as easy as it sounds if it needs to work in non-ksh93 shells

function capit { 
    typeset -u first
    first=${1%"${1#?}"}
    printf "%s\n" "${first}${1#?}"
    return 0
}

or maybe more neatly done in Perl…
This will capitalise each word in a sentence passed into it

function capit { 
    echo "$1" |  perl -pe "s/([\w']+)/\u\L\1/g"
    return 0
}

Remove / rename a directory / file containing weird control characters

Use ls with -i to see inode listing

ls -bali

Use find with -inum to get the filename and -exec to remove it

find . -inum <inode from ls listing> -exec rm -f {} \;

Run a local script on a remote host

Doing it this way means we do not need to scp the shell over to the host before executing it!

ssh user@host 'sh' < your_script.sh

First of all, this command is a redirection: your shell will open the file your_script.sh and feed it as input to the ssh command. ssh, in turn, will tunnel its stdin to the remote command, namely, sh instance. And sh without arguments reads its script from stdin. So we got sh instance, which is launched on remote host, but reads commands from your local file.

Get a list of running instances - but only those started by the current user

This is the way I wanted to do it but there's an issue. Where does that spare } character come from? Anyone?

ps -ef | grep [p]mon | awk -v dbowner=$(id -un) '{ if ($1==dbowner) { gsub(/ora_pmon_/,"",$NF); print $NF; } }'

ps listing

$ ps -ef | grep [p]mon
  oracle 13304024        1   0   Jun 07    # 2:39 ora_pmon_reportk
  oracle 26018178        1   0   Jun 07    # 3:01 ora_pmon_dwh_perf
  oracle 29229432        1   0   Jun 07    # 2:30 ora_pmon_adso
 oraebso 18022994        1   0   Jun 07    # 2:38 ora_pmon_EBSO
  oracle 30278192        1   0   Jun 07    # 2:48 ora_pmon_owb112k

Results of above command

reportk
dwh_perf
adso
}
owb112k

Workaround 1. Send the ps listing to a file and work on it without pipes. Works but it's a bit long-winded especially as we have to clean up afterwards.

ps -ef | grep [p]mon>/tmp/results.$$ 2>/dev/null; awk -v dbowner=$(id -un) '{ if ($1==dbowner) { gsub(/ora_pmon_/,"",$NF); print $NF; } }' /tmp/results.$$; rm -f /tmp/results.$$

Workaround 2. Don't like the grep -v but until I find out where that } is coming from..

ps -ef | grep [p]mon | awk -v dbowner=$(id -un) '{ if ($1==dbowner) { gsub(/ora_pmon_/,"",$NF); print $NF; } }' | egrep -v 'grep|}|ASM'

Found it. ps is particular. Cannot use ps -ef as the input to commands like awk. Do it like this…

ps -ef | grep [p]mon|while read line; do awk -v dbowner=$(id -un) '{ if ($1==dbowner) { gsub(/ora_pmon_/,"",$NF); print $NF; } }';done

Also see bash pitfalls

Return a list of Oracle databases running on a list of remote servers

#!/bin/ksh

serverlist=`cat /home/tools/ini/system/oracle_servers | sort -n | tr "\n" " "`

if [ -z "${serverlist}" ]
then
    echo "no servers found" && exit 1
fi


for server in ${serverlist}
do
    ssh ${server} "ps -ef | grep [o]ra_pmon_" >/tmp/${server}.dblist
done

for server in ${serverlist}
do
    cat /tmp/${server}.dblist | awk -F_ -v SRV=${server} 'BEGIN {print SRV ":"} {print $NF}' | tr "\n" " "
    echo
done

Clever trick to check whether SQL output needs sending to someone

Using an exit status from SQL*Plus back to the shell so we can decide what to do. Nice one!

#!/bin/bash
tempfile=/tmp/audit_locked_accounts_$ORACLE_SID.txt

# Start sqlplus and check for locked accounts
sqlplus -S "/ as sysdba" << EOF > $tempfile
set pages 0
select 'The following accounts were found to be unlocked and should not be'
from dual;

def exit_status = 0
col xs new_value exit_status

select username
,      account_status
,      1 as xs
from   dba_users
where  account_status != 'LOCKED'
and    username in ('HR', 'SCOTT', 'OUTLN', 'MDSYS', 'CTXSYS')
/
exit &exit_status
EOF

# If the exit status of sqlplus was not 0 then we will send an email
if [[ $? != 0 ]]; then
    mail -s "Accounts Unlocked in $ORACLE_SID" oracle < $tempfile
fi

Check RMAN logfiles for errors from cron every day

00 09 * * * /home/tools/scripts/oracle/dosh 'find /home/tools/logs/rman -name "*online.log" -mtime -1 -exec sed -ne "/^RMAN-/,/^$/p" {} \\; -ls' | mailx -s 'RMAN errors last night' [email protected]

/home/ibmtools/scripts/oracle/dosh 'find /oracle/export -name "expdp*log" -mtime -1 -exec grep ORA- {} \; -ls' | mailx -s 'Datapump errors for Baxter last night' [email protected]

Standard getopts

Only allows single character parameter keys.

while getopts "vf:" flag
do
    case "$flag" in
    v) VERBOSE=1;;
    f) FILE2RUN=$OPTARG;;
    esac
done
shift $((OPTIND-1))

An alternative to getopts

These methods have the advantage of allowing arguments of any length not just 1 character

# Parse command-line arguments
INST_ARG=
DO=
while [ $# -gt 0 ]
do
    case $1 in
        -force)
            FORCE=yes
            shift
            ;;

        -vm)
            echo
            echo "*** The \"-vm\" option has been deprecated."
            echo "***"
            echo "*** Now there is only one set of services running for all virtual machines."
            echo "*** If you meant to address a specific installation *instance*, please"
            echo "*** use the \"-instance\" option. If you meant to do smth to services running on"
            echo "*** a specific virtual machine, then don't pass any option at all."
            echo
            exit 1
            ;;

        -instance|-focus)
            shift
            if [ $# -eq 0 ]
            then
                echo
                echo "*** Instance name expected after \"-instance\" option."
                echo
                exit 1
            fi
            INST_ARG=$1
            shift
            ;;

        -all)
            INST_ARG=all
            shift
            ;;

        add|-add|+)
            DO="add"
            shift
            PKGIDS="$1"
            [ x$PKGIDS = x ] || shift
            ;;

        start|-start)
            DO="start"
            shift
            ;;

        stop|-stop)
            DO="stop"
            shift
            ;;

        restart|-restart)
            DO="restart"
            shift
        gui@*|-gui@*)
            GUI_CS="`echo $1 | awk -F\@ '{print $2}'`"
            DO="gui"
            shift
            ;;

        registry|-registry|reg|-reg)
            DO="registry"
            shift
            ;;

        -fuse)
            shift
            if [ $# -eq 0 ]
            then
                echo
                echo "*** Instance name expected after \"-fuse\" option."
                echo
                exit 1
            fi
            FUSE_INST_ARG=$1
            shift
            ;;

        shell|-shell)
            DO="shell"
            shift
            ;;

        direct|-direct|systemctl)
            NO_REDIRECT_TO_SYSTEMCTL="YES"
            shift
            ;;

        *)
            echo
            echo "*** Unknown argument \"$1\" encountered."
            echo
            exit 1
            ;;
    esac
done

another one…

countArgs=$#
while [[ $# -gt 0 ]]
do
    idx1=$($ECHO $1|$GREP "="|wc -l)
    if [[ $idx1 -gt 0 ]]; then
        key=$($ECHO $1|$CUT -d '=' -f 1)
        value=$($ECHO $1|$CUT -d '=' -f 2)
    else
        key=$($ECHO $1|$CUT -d '=' -f 1)
        idx2=$($ECHO $1|$GREP "^-"|wc -l)
        if [[ $idx2 -eq 0 ]]; then
            $ECHO -e "\n ERROR: $1 is an unsupported argument passed to agentDeploy.sh.\n"  
            usage
            exit 1
        fi
    fi

    case "$key" in
    -help)
        if [[ $upgradeFlag ]]; then
            upgradeUsage
        else
            freshUsage
        fi
        exit 0;;
    AGENT_BASE_DIR)
        agentBaseDir=$($ECHO $value|$SED 's/\/$//')
        checkBaseFlag=TRUE;;
    OMS_HOST)
        omsHost=$value
        checkOMSHost=TRUE;;
    EM_UPLOAD_PORT)
        omsPort=$value
        checkOMSPort=TRUE;;
    AGENT_INSTANCE_HOME)
        instHome=$($ECHO $value | $SED 's/\/$//');;
    AGENT_REGISTRATION_PASSWORD)
        pswd=$value;;
    s_encrSecurePwd)
        pswd=$value;;  
    RESPONSE_FILE)
        rspFlag=TRUE  
        rspLoc=$value;;
    OLD_AGENT_VERSION)
        oldAgentVersion=$value;;            
    OLDHOME)
        oldHome=$value;;    
    -debug)
        debugSwitch=TRUE;;
    -forceConfigure)
        forceFlag=TRUE;;
    -configOnly)
        configFlag=TRUE    
        validationFlag=TRUE;;
    -agentImageLocation)
        archiveLoc=$value
        checkArchiveFlag=TRUE;;
    -invPtrLoc) shift
        ptrLoc=$1;;
    -Upgrade)
        UpgradeFlag=TRUE
        validFlag=TRUE;;
    INVENTORY_LOCATION)
        validFlag=TRUE;;
    b_forceInstCheck)
        validFlag=TRUE
        forcefullFlag=TRUE;;
    -prereqOnly)
        validationFlag=TRUE
        prereqFlag=TRUE;;
    -executeRootsh)
        validFlag=TRUE;;
    *)  idx=$($ECHO $1|$GREP "^-"|wc -l)
        if [[ $idx -eq 0 ]]; then
            validFlag=TRUE
        else
            $ECHO -e "\n ERROR: Invalid argument $key passed."
            usage
            exit 1          
        fi
    esac
    shift
done

getopts - another way - found in /etc/init.d/functions (daemon function) in Fedora

while [ "$1" != "${1##[-+]}" ]; do
        case $1 in
        '')
            echo $"$0: Usage: daemon [+/-nicelevel] {program}" "[arg1]..."
            return 1
            ;;
        --check)
            base=$2
            gotbase="yes"
            shift 2
            ;;
        --check=?*)
            base=${1#--check=}
            gotbase="yes"
            shift
            ;;
        --user)
            user=$2
            shift 2
            ;;
        --user=?*)
            user=${1#--user=}
            shift
            ;;
        --pidfile)
            pid_file=$2
            shift 2
            ;;
        --pidfile=?*)
            pid_file=${1#--pidfile=}
            shift
            ;;
        --force)
            force="force"
            shift
            ;;
        [-+][0-9]*)
            nice="nice -n $1"
            shift
            ;;
        *)
            echo $"$0: Usage: daemon [+/-nicelevel] {program}" "[arg1]..."
            return 1
            ;;
      esac
    done

getopts - yet another way - found in adstrtal.sh (middleware start script for EBS)

#
# Parse Arguments
#

for nxtarg in $*
do
  arg=`echo $nxtarg | sed 's/^//'`

  case $arg in
    -secureapps)    if test "$secureapps" = "" ; then
                       secureapps=$arg
                    else
                       echo "$0: Duplicate Argument passed : $arg"
                       usage
                    fi
                ;;
    -nodbchk)       if test "$nodbchk" = "" ; then
                       nodbchk=$arg
                    else
                       echo "$0: Duplicate Argument passed : $arg"
                       usage
                    fi
                ;;
    -nopromptmsg)   if test "$nopromptmsg" = "" ; then
                       nopromptmsg=$arg
                    else
                       echo "$0: Duplicate Argument passed : $arg"
                       usage
                    fi
                ;;
    *)              if test "$unpw" = "" ; then
                       unpw=$arg
                    else
                       echo "$0: Duplicate Argument passed : $arg"
                       usage
                    fi
  esac
done

getopts - and another in adautocfg.sh

for myarg in $*
do

  arg=`echo $myarg | sed 's/^-//'`
  case $arg in
    appspass=*)
            appspass=`echo $arg | sed 's/appspass=//g'`
            shift
            ;;
    nocustom)
            myparams="$myparams $arg"
            shift
            ;;
    noversionchecks)
            myparams="$myparams $arg"
            shift
            ;;
    promptmsg=*)
            promptmsg=`echo $arg | sed 's/promptmsg=//g'`
            shift
            ;;
        *)  echo "$0: unrecognized action specified"
            exit 1
  esac
done

getopts - ok, this is the last one (from ${OMS_HOME}/OMSPatcher/wlskeys/createkeys.sh

### parse ###
oh=${ORACLE_HOME}
location=
getoh=0
getloc=0
gethelp=0

for arg in $@
do
    if [ $getoh = 1 ]; then
        oh=$arg
        getoh=2
    fi
    if [ $getloc = 1 ]; then
        location=$arg
        getloc=2
    fi

    if [ "$arg" = "-oh" ]; then
        getoh=1
    fi
    if [ "$arg" = "-location" ]; then
        getloc=1
    fi
    if [ "$arg" = "-help" ]; then
        gethelp=1
    fi
done

### help ###
if [ $gethelp = 1 ] || [ -z "${oh}" -o -z "${location}" ]; then
    echo "Usage:"
    echo "      ${binname} [-help]"
    echo "      ${binname} -oh <OMS Platform home> -location <location to store the config and key files>"
    echo ""
    echo "                 -oh       OMS Platform home path"
    echo "                 -location Location to store the configuration and key files"
    exit 1
fi

Run a script on multiple servers

#!/usr/bin/env ksh
# ==============================================================================
# Name         : dosh   (Distributed Oracle SHell)
# Description  : Runs a command on all Oracle servers
#
# Parameters   : h - displays help
#                b - execute in the background (in parallel)
#                v - verbose (default, like all unix commands is silent)
#                c - command to be executed
#                f - file containing commands to be executed
#
# Example      : ./dosh -v -c 'ls -al'
#                ./dosh -v -f /tmp/complex_shell
#
# Notes        : Escape " and $ with \
#
# Modification History
# ====================
# When      Who               What
# ========= ================= ==================================================
# 08-FEB-13 Stuart Barkley    Created
# 08-JUL-15 Stuart Barkley    Background execution mode
# ==============================================================================
# --------------------------------------------------------------------------
# this list of servers is generated by the DBAHAWK tool so will be up to date
# --------------------------------------------------------------------------
serverfile=/oracle/scripts/.shb/oracle_servers
[[ -z "${serverfile}" ]] && echo "Server list ${serverfile} not found" && exit 1

serverlist=`cat ${serverfile} | sort -n | tr "\n" " "`


# -------------------------
# get the arguments, if any
# -------------------------
while getopts "hbvc:f:" OPT
do
    case "$OPT" in
    h) echo "\nUsage: $0 [-b] [-v] [-h] -c '<command_to_be_executed_remotely>'\n";
       exit;
       ;;
    b) BACKGROUND="Y";
       ;;
    v) VERBOSE="Y";
       ;;
    c) CMMND=$OPTARG;
       ;;
    f) CMDFILE=$OPTARG;
       ;;
    *) echo "\nUsage: $0 [-b] [-v] [-h] [-c '<command_to_be_executed_remotely>' | -f <file to execute>]\n";
       exit;
       ;;
    esac
done
shift $((OPTIND-1))
VERBOSE=${VERBOSE:-"N"}


# --------------------------------
# check we have required arguments
# --------------------------------
[[ -z $CMMND && -z $CMDFILE ]] && printf "%s\n" "Usage: $0 [-b] [-v] [-h] [-c '<command_to_be_executed_remotely>' | -f <file to execute>]" && exit 1


# ---------------------------------
# put the thing to be run in a file
# ---------------------------------
if [[ ! -z $CMMND ]]; then
    printf "%s\n" "$CMMND" > /tmp/dosh.$$
else
    cp $CMDFILE /tmp/dosh.$$
fi


# ----------------------------
# loop over the remote servers
# ----------------------------
for server in ${serverlist}
do
    if [[ "$VERBOSE" == "Y" ]]; then
        printf "%s %s\n" "${server}" "$(date '+%d-%b-%Y %H:%M:%S')"
        printf "%s\n" "-------------------------------------------"
    fi
    scp -q /tmp/dosh.$$ ${server}:/tmp/
    [[ $? -ne 0 ]] && echo "WARNING: Failed to connect to ${server}. Are ssh keys setup?, skipping this server" && continue
    if [[ "$BACKGROUND" == "Y" ]]; then
        ssh -n -o BatchMode=yes ${server} "ksh /tmp/dosh.$$; rm -f /tmp/dosh.$$"> /tmp/dosh_${server}.out 2>&1 &
    else
        ssh -n -o BatchMode=yes ${server} "ksh /tmp/dosh.$$; rm -f /tmp/dosh.$$"
    fi
    [[ "$VERBOSE" == "Y" ]] && printf "\n\n"
done
rm -f /tmp/dosh.$$

[[ "$BACKGROUND" == "Y" ]] && printf "%s\n" "Type 'ls -altr /tmp/dosh*out' to see output from the commands"

Get a stack trace a running process on Linux

 pstack <PID>

Get a stack trace a running process on AIX

 procstack <PID>

Trace a running process on Linux

 strace -frT -o /tmp/strace-output.$$ -p <PID>

Trace a running process on AIX

 truss -rall -wall -xall -sall -p <PID>
or
truss -aefdDo /tmp/truss_output_$$ -p <PID>

Get AIX info

Borrowed from /opt/commvault/Base/get_sys_info_AIX.sh

#!/bin/ksh

CVREG="/etc/CommVaultRegistry"
INSTANCE="Instance001"
BASEDIR=`grep -w dBASEHOME $CVREG/Galaxy/$INSTANCE/Base/.properties | awk '{print $2}'`
. $BASEDIR/common_info_funcs.sh
print_header

echo "List of Mountpoints"
mount 2>&1
print_line

echo "System Information (lsconf o/p)"
lsconf 2>&1
print_line

echo "Disk Information"
lsdev -Cc disk 2>&1
print_line
for disk in `lsdev -Cc disk -r name`
do
    echo "Attributes for $disk"
    lsattr -El $disk 2>&1
    print_line

    echo "lscfg -vl $disk"
    lscfg -vl $disk 2>&1
    print_line
done

echo "LV Information"
lsdev -Cc 'logical_volume' -s lvsubclass  2>&1
print_line

for lv in `lsdev -Cc 'logical_volume' -s lvsubclass -r name`
do

    echo "Detailed info for $lv"
    lslv $lv 2>&1
    print_line

    echo "Attributes for $lv"
    lsattr -El $lv 2>&1
    print_line

    echo "PV mapping for $lv"
    lslv -m $lv
    print_line
done



exit 0

Find hardware information on AIX

lparstat -i

CPU, memory and VG info etc.

lsconf

or a quicker way to find the Machine serial number…

odmget CuAt | grep -p systemid | awk '$1 == "value" { print substr($3,8,7) }'

How can a shell script find out what directory it is in?

basename $0 does not always give the desired answer (might give “./”)!

DIR="$(cd "$(dirname "$0")" && pwd)"

Array processing

array=(1 2 3)
unset array[2]
echo ${array[2]}          # null
indices=(${!array[@]})    # create an array of the indices of "array"
size=${#indices[@]}       # the size of "array" is the number of indices into it
size=${#array[@]}         # same
echo ${array[@]: -1}      # you can use slices to get array elements, -1 is the last one, etc.
for element in ${array[@]}; do    # iterate over the array without an index

for index in ${indices[@]}        # iterate over the array WITH an index
do
    echo "Index: ${index}, Element: ${array[index]}"
done

for index in ${!array[@]}         # iterate over the array WITH an index, directly

array+=("new element")    # append a new element without referring to an index
((counter++))             # shorter than ((counter=counter+1)) or ((counter+=1))
if [[ $var == 3 ]]        # you can use the more "natural" comparison operators inside double square brackets
while [[ $var < 11 ]]     # another example
echo ${array[${index}-1]  # math inside an array subscript

Send an email by talking directly to an smtp server

#!/bin/bash
telnet smtp.domain.com 25 <<EOTXT>>/tmp/smtp.log
HELO me.domain.com
MAIL FROM:<[email protected]>
RCPT TO:<[email protected]>
DATA
From: Stuart <[email protected]>
To: Anne <[email protected]>
Subject: testing smtp email

Hello, this should appear in the body
.
QUIT
EOTXT

Send an email by talking directly to an smtp server via file descriptor (no telnet! this time), adding authentication

#!/bin/bash

#
# mail.sh
#
# 2008 - Mike Golvach - [email protected]
# 2010 - Rayber
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#

if [ $# -ne 7 ]
then
echo "Usage: $0 FromAdress ToAdress Domain MailServer MailTextFile AuthEmail AuthPass"
exit 1
fi

from=$1
to=$2
domain=$3
mailserver=$4
mailtext=$5
authemail=`echo $6|openssl enc -base64|awk 'sub("..$", "")'`
authpass=`echo $7|openssl enc -base64|awk 'sub("..$", "")'`

if [ ! -f $mailtext ]
then
echo "Cannot find your mail text file. Exiting..."
exit 1
fi

exec 9<>/dev/tcp/$mailserver/25
echo "HELO $domain" >&9
read -r temp <&9
echo "$temp"
echo "auth login" >&9
read -r temp <&9
echo "$authemail" >&9
read -r temp <&9
echo "$authpass" >&9
read -r temp <&9
echo "Mail From: $from" >&9
read -r temp <&9
echo "$temp"
echo "Rcpt To: $to" >&9
read -r temp <&9
echo "$temp"
echo "Data" >&9
read -r temp <&9
echo "$temp"
cat $mailtext >&9
echo "." >&9
read -r temp <&9
echo "$temp"
echo "quit" >&9
read -r temp <&9
echo "$temp"
9>&-
9<&-
echo "All Done Sending Email. See above for errors"
exit 0

Send an html file to an email recipient from a Unox server using sendmail

(
echo "From: $(id -un)@$(hostname)";
echo "To: [email protected]";
echo "Subject: Your HTML report";
echo "Content-Type: text/html";
echo "MIME-Version: 1.0";
cat /file/with/html/tags.html;
) | sendmail -t

rsync examples

#!/bin/sh
ssh  <remote_host> '/bin/mkdir -p /etc /etc/rc.config.d /etc/security /etc/mail'
rsync --rsync-path /usr/bin/rsync -Liprogtz --out-format=%f%L  /etc/passwd /etc/passwd.post /etc/group /etc/hosts /etc/services /etc/resolv.conf /etc/exclude.rootvg <remote_host>:/etc
rsync --rsync-path /usr/bin/rsync -Liprogtz --out-format=%f%L  /etc/hosts.allow.xcat <remote_host>:/etc/hosts.allow
rsync --rsync-path /usr/bin/rsync -Liprogtz --out-format=%f%L  /etc/rc.config.d/sap <remote_host>:/etc/rc.config.d
rsync --rsync-path /usr/bin/rsync -Liprogtz --out-format=%f%L  /etc/security/group /etc/security/limits /etc/security/login.cfg /etc/security/passwd /etc/security/user <remote_host>:/etc/security
rsync --rsync-path /usr/bin/rsync -Liprogtz --out-format=%f%L  /etc/mail/sendmail.cf <remote_host>:/etc/mail

rsync -av --progress /home/ibmtools/scripts/oracle/* benouerp07:/home/ibmtools/scripts/oracle/
rsync -avzh --progress --exclude='*dmp' --exclude='*csv' oracle@hn481:/oracle/scripts /Users/stuartb/oracle/

Handy aliases

Strip out comments and blank lines from a file

alias strip='grep -Ev '\''^(#|$)'\'''

Does a ps and a grep

alias psg='ps -ef | grep -v $$ | grep -i '

Getting a decent listing of filesystem space available. It is ordered such that the filesystems with no space left are at the end.

OS=$(uname -s)
[[ "$OS" == "SunOS" ]] && alias dfg='df -h|sed -e '1d'|sort -n -k5|awk '\'' BEGIN {printf("%-35s%-10s%-10s%-6s%-30s\n","Filesystem","Total","Free","%Used","Mounted on")} {printf("%-35s%-10s%-10s%-6s%-30s\n",$1,$2,$4,$5,$6)}'\'''
[[ "$OS" == "AIX" ]]   && alias dfg='df -g|sed -e '1d'|sort -n -k4|awk '\'' BEGIN {printf("%-35s%-10s%-10s%-6s%-30s\n","Filesystem","Total GB","Free","%Used","Mounted on")} {printf("%-35s%-10s%-10s%-6s%-30s\n",$1,$2,$3,$4,$7)}'\'''

A handy way of listing subdirectories and their files

alias filetree="find . -print | sed -e 's;[^/]*/;|____;g;s;____|; |;g'"

Watch progress of a copy

alias cpProgress="rsync --progress -ravz"

Reboots Linksys router

alias rebootlinksys="curl -u 'admin:password' 'http://192.168.1.2/setup.cgi?todo=reboot'"

Nice one for bash. Colour codes the prompt depending on the outcome of the previous command

bash_prompt_command()
{
    RTN=$?
    prevCmd=$(prevCmd $RTN)
}
PROMPT_COMMAND=bash_prompt_command
prevCmd()
{
    if [ $1 == 0 ] ; then
        echo $GREEN
    else
        echo $RED
    fi
}
if [ $(tput colors) -gt 0 ] ; then
    RED=$(tput setaf 1)
    GREEN=$(tput setaf 2)
    RST=$(tput op)
fi
export PS1="\[\e[36m\]\u.\h.\W\[\e[0m\]\[\$prevCmd\]>\[$RST\]"

Mmmm, to be looked into. Executes remote commands on a unix box using curl.

#/bin/sh
#
# WAG320N-HACK
# Ver. 1.0
# 12/09/2010
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program.  If not, see <http://www.gnu.org/licenses/>.

# Set username and password in the form of "username:password"
# example: "admin:admin"
my_access="admin:admin"

# Parameters test
if [ -z "$1" ]
then
  echo "wag320n-hack.sh: missing remote command"
  echo "usage: wag320n-hack.sh <remote command>"
  echo "example: wag320n-hack.sh /bin/ls -la /usr/sbin"
  echo "Note: always use full path"
  echo ""
  echo "wag320n-hack.sh - Ver. 1.0 - 12/09/2010"
  echo "Licensed under GPL V. 3"
  echo ""
  exit 0
fi

# Get the command
my_command="ping_size="'$('"$@"' 1>&2)'

curl -s -G -u "$my_access" --data-urlencode 'todo=ping_test' --data-urlencode 'this_file=Diagnostics.htm' --data-urlencode 'next_file=Ping.htm' --data-urlencode 'c4_ping_ipaddr=192.168.1.1' --data-urlencode 'ping_timeout=5000' --data-urlencode 'ping_interval=1000' --data-urlencode 'ping_number=1' --data-urlencode "$my_command" http://192.168.1.1/setup.cgi | sed -e '/HTTP\/1.0 200 OK/q'

How to configure SSH with public/private keys

unixpeople.com

Use SSH config file

Host server10
  Hostname 1.2.3.4
  IdentityFile ~/.ssh/id_dsa
  user foobar
  Port 30000
  ForwardX11Trusted yes
  TCPKeepAlive yes

then just connect using

ssh server10

A decent sed tutorial

From grymoire.com

A decent korn/bash shell tutorial

From dartmouth.edu Reproduced here just in case it disappears! Advanced shell scripting

trap

     Example Handling Traps With ksh - Discussion of the kill command


EXAMPLE TEMPLATE:


PRODUCT:    HP-UX 11iV1 Version B.11.11
            HP Tru64 V5.1B PK4
            Sun/Solaris SunOS V5.8
            Linux 2.6 kernel


COMPONENT:  ksh


SOURCE:     Philippe Vouters
            Fontainebleau/France


LOW COST HIGH-TECH PRODUCTS:  http://techno-star.fr


OVERVIEW:

The ksh script below shows how to eventually handle traps in the situation
where someone might try to kill a script by killing individual commands run
by that script or the entire process group a script is running in. The kill
command (usually a shell builtin) may be used to send a signal to a process
group (with the -<pid> syntax) or an individual process. The example ksh
script below runs /bin/sleep as the foreground process, the example ksh
scripts immediately returns when the /bin/sleep process has terminated. Most
signals sent to the shell are ignored until after the foreground process
terminates. This is in order to avoid creating zombie processes. Therefore a
kill <pid> on the example ksh script waits for the termination of the
/bin/sleep process.

The status value $? in the trap refers to the exit status of the command to run
and therefore is the exit status of the /bin/sleep process. The called function
in the trap handler shows how to correctly examine the effect of the kill
command on the shell or it's children.

To examine the value of $? in a trap handler means that you must understand what
it can be set and how different signals delivered to either the shell or the
foreground process (or the process group) might affect the value of $?.

The example shell script prints $? using echo but it does not perform tests on
the value of $?. For a complete solution when attempting to trap signals in a
shell you would also need code that examined the value of $? after the
foreground process had completed.


      *  CAUTION ***

This sample script has been tested using HP-UX B.11.11, HP Tru64 V5.1B PK4,
SunOS V5.8 and Fedora Core 4 (homed version of Red Hat Linux).  However, we
cannot guarantee its effectiveness because of the possibility of error in
transmitting or implementing it. It is meant to be used as a template for
writing your own scripts, and may require modification for use on your system.


SCRIPT NOTES:

To notice that the ksh script and /bin/sleep share the same process group
identifier (PGID), issue the following commands:

[philippe@victor ~]$ who
philippe :0           Jan 10 10:16
philippe pts/1        Jan 10 21:30 (:0.0)
philippe pts/2        Jan 10 21:30 (:0.0)
[philippe@victor ~]$ tty
/dev/pts/1
[philippe@victor ~]$ ps -j -t pts/2
  PID  PGID   SID TTY          TIME CMD
11072 11072 11072 pts/2    00:00:00 bash
11113 11113 11072 pts/2    00:00:00 ksh
11116 11113 11072 pts/2    00:00:00 sleep

In this case sending kill -INT -11113 will send SIGINT to the process group
11113. Both of the ksh and sleep processes are contained within this process
group.

Important Note:

On HP-UX, you have to $ export UNIX95=1 in order to be able to use the
-j option of the ps command.


SCRIPT:

                             COPYRIGHT (C) 2005 BY
                              HEWLETT-PACKARD COMPANY
                                ALL RIGHTS RESERVED.

     THIS SOFTWARE IS FURNISHED UNDER A LICENSE AND MAY BE USED AND COPIED
     ONLY IN ACCORDANCE WITH THE TERMS OF SUCH LICENSE AND WITH THE INCLUSION
     OF THE ABOVE COPYRIGHT NOTICE.  THIS SOFTWARE OR ANY OTHER COPIES
     THEREOF MAY NOT BE PROVIDED OR OTHERWISE MADE AVAILABLE TO ANY OTHER
     PERSON.  NO TITLE TO AND OWNERSHIP OF THE SOFTWARE IS HEREBY TRANSFERRED.

     THE INFORMATION IN THIS SOFTWARE IS SUBJECT TO CHANGE WITHOUT NOTICE AND
     SHOULD NOT BE CONSTRUED AS A COMMITMENT BY HEWLETT-PACKARD COMPANY.

     HP ASSUMES NO RESPONSIBILITY FOR THE USE OR RELIABILITY OF ITS
     SOFTWARE ON EQUIPMENT THAT IS NOT SUPPLIED BY HP.

     NO RESPONSIBILITY IS ASSUMED FOR THE USE OR RELIABILITY OF SOFTWARE
     ON EQUIPMENT THAT IS NOT SUPPLIED BY HEWLETT-PACKARD COMPANY.

     SUPPORT FOR THIS SOFTWARE IS NOT COVERED UNDER ANY HP SOFTWARE
     PRODUCT SUPPORT CONTRACT, BUT MAY BE PROVIDED UNDER THE TERMS OF THE
     CONSULTING AGREEMENT UNDER WHICH THIS SOFTWARE WAS DEVELOPED.

#!/bin/ksh
function handle_signal
{
        print -n "pid $$ recieved $2 "
        if [[ $1 = 0 ]];then
            print but foreground command ended successfully
        else
                if [[ $1 = $3 ]];then
                    print and so did the last foreground command
                else
                    print -n "and the exit status of the last foreground "
                    print command was $1
                fi
        fi
        # Kill our process group and then ourselves with SIGTERM, giving a
        # pid of 0 sends the signal to our process group. Killing the process
        # group should kill us as well, this assumes that SIGTERM is not
        # handled by any process in the process group.
        #
        # This code could be replaced with an exit with an exit value that
        # would indicate what the problem was to the caller. That is replace
        # these two lines with:
        #
        # exit $3
        #
        # or a specific exit code could be used.
        #
        kill -TERM 0
        kill -TERM $$
        }
OS=$(uname -a | awk '{print $1}')
if [[ "$OS" = "Linux" ]]; then
    offset=256
elif [[ ("$OS" = "HP-UX") ||
        ("$OS" = "SunOS") ||
        ("$OS" = "OSF1") ]]; then
   offset=128
fi
trap 'RC=$?; handle_signal $RC SIGINT $offset+2' INT
trap 'RC=$?; handle_signal $RC SIGQUIT $offset+3' QUIT
/bin/sleep 20
echo $?

DNS not working

Ping to an IP address works

 ping 74.125.136.103

but this doesn't

 ping www.google.com

Check resolv.conf

 cat /etc/resolv.conf
 nameserver 95.130.132.17
 nameserver 95.130.132.18

I had changed internet provider and forgot to update this. Just to set it to the router address and let that do the resolution

 nameserver 192.168.1.1

Create new image with kvm

Ref: http://www.cyberciti.biz/faq/kvm-virtualization-in-redhat-centos-scientific-linux-6/
Build an empty space for a CentOS virtual machine

qemu-img create -f qcow2 centos.img 12G

Tried creating image with

sudo virt-install -n CentOS --description "Trying out CentOS" --ram=1024 --vcpus=1 --cpu host --hvm --cdrom /home/bey9at77/Downloads/c6-x86_64-20130910-1.qcow2 --graphics vnc --disk path=/var/lib/libvirt/images/centos.img,bus=virtio,size=10

gives error

Starting install...
Allocating 'centos.img'                                                                                                                                                                                               |  10 GB     00:00
ERROR    internal error Process exited while reading console log output: char device redirected to /dev/pts/1
qemu-kvm: -drive file=/home/bey9at77/Downloads/c6-x86_64-20130910-1.qcow2.bz2,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw: could not open disk image /home/bey9at77/Downloads/c6-x86_64-20130910-1.qcow2: Permission denied

rpm / yum commands

Install an RPM Package

rpm -ivh pidgin-2.7.9-5.el6.2.i686.rpm

-i - install
-v - verbose
-h - print progress hashes

Check dependencies of RPM Package before Installing

rpm -qpR BitTorrent-5.2.2-1-Python2.4.noarch.rpm

-q - query
-p - list package capabilities
-R - list dependent package capabilities

Install RPM Package with all dependencies

yum install BitTorrent-5.2.2-1-Python2.4.noarch.rpm

Install RPM Package with all dependencies (when RPM has been downloaded to local machine)

yum localinstall BitTorrent-5.2.2-1-Python2.4.noarch.rpm

Force Install a RPM Package without dependencies

Package will not work if dependencies are required

rpm -ivh --nodeps BitTorrent-5.2.2-1-Python2.4.noarch.rpm

Check an Installed RPM Package

rpm -q BitTorrent

List all files of an installed RPM package

rpm -ql BitTorrent

List All Installed RPM Packages

rpm -qa

Query information about an installed RPM package

rpm -qi vsftpd

Query information about a not yet installed RPM package

rpm -qip sqlbuddy-1.3.3-1.noarch.rpm

(Forcibly) Remove an RPM Package

Use package name (as seen in -qi above), not full name

rpm -ev (--nodeps) vsftpd

Query a file that was installed as part of an RPM Package (which package contained this file)

rpm -qf /usr/bin/htpasswd

Verify an RPM package

Compares information of installed files of the package against the rpm database

rpm -Vp sqlbuddy-1.3.3-1.noarch.rpm

Rebuild corrupted RPM database

cd /var/lib
rm __db*
rpm --rebuilddb
rpmdb_verify Packages

Install rpmforge repository

Download rpmforge-release-0.5.3-1.el6.rf.x86_64.rpm
Import the key

sudo rpm --import http://apt.sw.be/RPM-GPG-KEY.dag.txt

Install the repository

sudo rpm -i rpmforge-release-0.5.3-1.el6.rf.x86_64.rpm

Check the installation

rpm -K rpmforge-release-0.5.3-1.el6.rf.x86_64.rpm

Test it

sudo yum install terminator

Install rpmfusion repository

su -c 'yum localinstall --nogpgcheck http://download1.rpmfusion.org/free/el/updates/6/i386/rpmfusion-free-release-6-1.noarch.rpm http://download1.rpmfusion.org/nonfree/el/updates/6/i386/rpmfusion-nonfree-release-6-1.noarch.rpm'

config file for yum

Checkout this file for global yum config

/etc/sysconfig/yum-cron-background

Setup Oracle Enterprise Linux (RedHat) with yum server

You need to download the yum .repo file from the server, as per the steps below. After this, you need to enable a flag in the .repo file as per your operating system version. Having done these two steps, when you run yum install <pkgname> command on your linux box, the Oracle's yum server will be scanned, the dependent & the relevant rpm's will be download and installed for you.

 cd /etc/yum.repos.d

To download files here

 wget http://public-yum.oracle.com/public-yum-el5.repo

A file named public-yum-el5.repo will be created in your directory
Edit this file and enter enabled=1 against the operating systems which is relevant to you

 vi public-yum-el5.repo

Next run the yum command

 yum install package-name

To change to static IP address (Raspberry Pi)

As root:

cd /etc/networks
vi interfaces

replace the line “iface eth0 inet dhcp” with

iface eth0 inet static
address 192.168.1.100
netmask 255.255.255.0
gateway 192.168.1.1

You should also take a look at the file /etc/resolv.conf and check it has a nameserver entry (probably pointing at your default gateway) or direct to your ISP name servers.

nameserver 192.168.1.1

Troubleshoot wireless network problems

connect to wifi network from command-line in linux - from blackmoreops.com

Short summary of all the things you need to do in just few lines

root@kali:~# iw dev
root@kali:~# ip link set wlan0 up
root@kali:~# iw wlan0 scan
root@kali:~# wpa_passphrase blackMOREOps >> /etc/wpa_supplicant.conf
root@kali:~# wpa_supplicant -i wlan0 -c /etc/wpa_supplicant.conf
root@kali:~# iw wlan0 link
root@kali:~# dhclient wlan0
root@kali:~# ping 8.8.8.8
(Where wlan0 is wifi adapter and blackMOREOps is SSID)
(Add Routing manually)
root@kali:~# ip route add default via 10.0.0.138 dev wlan0

To change to static IP address (Redhat/CentOS)

As root:

vi /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
BOOTPROTO=STATIC
IPADDR=192.168.1.111
NETMASK=255.255.255.0
GATEWAY=192.168.1.1
ONBOOT=yes
DNS1=8.8.8.8
DNS2=8.8.4.4

Resrart the network interface

/etc/init.d/network stop
/etc/init.d/network start
or
service network restart

Check name server entry in resolv.conf

vi /etc/resolv.conf
nameserver 192.168.1.1

Enable processes / services to start at boot time

sudo chkconfig httpd on
sudo chkconfig mysqld on

Run a command on lots of servers in parallel

This is a damn fine AIX utility - part of the CSM Distributed Shell.

 dsh -a "ls -al /etc/apache2/*conf"

will list the Apache configuration file on all reachable servers (nodes)

Download a gzip file and pipe it into tar

cd ~ && wget -O - "https://www.dropbox.com/download?plat=lnx.x86_64" | tar xzf -

Check in a script to make sure it is run only by root

RUID=`/usr/bin/id|$AWK -F\( '{print $1}'|$AWK -F\= '{print $2}'`
if [ ${RUID} != "0" ];then
   $ECHO "This script must be executed as root"
   exit 1
fi

Set terminal to use Backspace key to erase previous character instead of Control-H

Been looking for this for a long time.
You can put:

stty erase <CTRL-V><Backspace key>

in your .profile but this will be ruined if you do a copy/paste into another file.
I wanted a way of doing this without entering the control character in the .profile. Finally stumbled upon it. And it's so simple. Just escape the caret!

stty erase \^?

Put this in the .profile. It's copy/pastable and it works!
If you want CTRL-H to be your erase character, just do this:

stty erase \^H

Play with the terminal settings and reset them again either side of requesting a password

The -g option of stty gives a compact list of all the settings or the terminal and can be used as input to stty

OLDCONFIG=`stty -g`          # save terminal configuration
stty -echo                   # turn character echoing off
echo "Enter password: \c"
read PASSWD                  # get the password
stty $OLDCONFIG              # restore terminal configuration

Reset terminal to "sane" characteristics

If you've done a cat of a binary file or something else weird and your terminal is left in a mess, the following key sequence should bring it back to normal

<CTRL-J>stty sane<CTRL-J>

Install OpenOffice on RedHat Enterprise when yum install doesn't!

Download Package

wget http://sourceforge.net/projects/openofficeorg.mirror/files/4.0.1/binaries/en-US/Apache_OpenOffice_4.0.1_Linux_x86-64_install-rpm_en-US.tar.gz/download -O Apache_OpenOffice_4.0.1_Linux_x86-64_install-rpm_en-US.tar.gz

Change to root

sudo su -

Remove the old stuff

yum remove openoffice* libreoffice*

Extract Package

tar -xvf Apache_OpenOffice_4.0.1*
cd en-US

Install Package and exit root

rpm -Uvh RPMS/*.rpm RPMS/desktop-integration/openoffice4.0-redhat-*.rpm
exit

Start it

openoffice4

What does this do?

while IFS= read -r line; do
    echo "[$(date "+%F %T")] - $line"
done < <(iwevent)

or

while IFS= read -r line; do
    printf "%s\n%s\n" "$line" "Yohooo! One more package."
done < <(tcpdump -i any -nS)

List of Special Characters and what they mean

From Bruce Barnett

Character	Where	Meaning
<RETURN>	csh, sh	Execute command
#	csh, sh, ASCII files	Start a comment
<SPACE>	csh, sh	Argument separator
`	csh, sh	Command substitution
"	csh, sh	Weak Quotes
'	csh, sh	Strong Quotes
\	csh, sh	Single Character Quote
variable	sh, csh	Variable
variable	csh, sh	Same as variable
|	csh, sh	Pipe character
^	sh	Pipe Character
&	csh, sh	Run program in background
?	csh, sh	Match one character
*	csh, sh	Match any number of characters
;	csh, sh	Command separator
;;	sh	End of Case statement
~	csh	Home Directory
~user	csh	User's Home Directory
!	csh	History of Commands
-	Programs	Start of optional argument
$#	csh, sh	Number of arguments to script
$*	csh, sh	Arguments to script
$@	sh	Original arguments to script
$-	sh	Flags passed to shell
$?	sh	Status of previous command
$$	sh	Process identification number
$!	sh	PID of last background job
&&	sh	Short-circuit AND
||	sh	Short-circuit OR
.	csh, sh	Typ. filename extension
.	sh	Source a file and execute as command
:	sh	Nothing command
:	sh	Separates Values in environment variables
:	csh	Variable modifier
Character	Where	Meaning
[ ]	csh, sh	Match range of characters
[ ]	sh	Test
%job	csh	Identifies job Number
(cmd;cmd)	csh. sh	Runs cmd;cmd as a sub-shell
{ }	csh	In-line expansions
{cmd;cmd }	sh	Like (cmd;cmd ) without a subshell
>ofile	csh, sh	Standard output
>>ofile	csh, sh	Append to standard output
<ifile	csh, sh	Standard Input
<<word	csh, sh	Read until word, substitute variables
<<\word	csh, sh	Read until word, no substitution
<<-word	sh	Read until word, ignoring TABS
>>!file	csh	Append to file, ignore error if not there
>!file	csh	Output to new file, ignore error if not there
>&file	csh	Send standard & error output to file
<&digit	sh	Switch Standard Input to file
<&-	sh	Close Standard Input
>&digit	sh	Switch Standard Output to file
>&-	sh	Close Standard Output
digit1<&digit2	sh	Connect digit2 to digit1
digit<&-	sh	Close file digit
digit2>&digit1	sh	Connect digit2 to digit1
digit>&-	sh	Close file digit

Table of Contents

Find all database-related files from the controlfile

List files using find without recursively descending into subdirectories

Print shell commands as they are being run

How to assign a certain keyboard shortcut to paste specific item

A poor man's 'expect'

Remove a directory from the PATH variable

Random password generator for SQL*Plus or PL/SQL

Inventory commands for AIX

Monitoring commands for AIX

AIX hacmp (PowerHA) startup and shutdown scripts

The app_start.sh script referenced above

Other userful cluster commands

cat a local file to the end of a file on a remote host over ssh

Countdown timer loop

Sort a comma separated list in shell

Generate a random sequence of characters (maybe for a password)

How to automatically download and process email attachments from Gmail

Useful references

Install tools

Configure fetchmail

Configure maildrop

uudeview

Reset terminal to a sane state if it has got messed up

The underused xargs command!

Watch a directory and automatically perform an action when new files come into it

Top 10 (or 20) thread count consumers in AIX

How much memory is being used on AIX?

Top memory consumers in AIX

Top 10 memory-consuming processes

Show paged memory hogs on AIX

Top 10 processes in order of I/O usage

Top 10 CPU consuming processes

How can I either encrypt or render my shell script unreadable?

Date arithmetic

Out of memory saving lines for undo - try using ed

Restart a program / service / daemon if it has not been seen for a while

Start / Stop NFS mounts

Start / Stop / List Samba shares

Script called at node startup using a file on a shared cluster filesystem to start Oracle databases and listeners

Script called by previous one to start databases and / or listeners (supplied by Oracle Corp. many eons ago

Use sed to repeat characters

Convert between lower and upper case using ansi standard

Use sed to delete the first line of a file or command output printing from the second line onwards

Use sed to munge a controlfile into submission!

Find all files with lines containing a certain text

Find all files where a certain text does not exist

tar and gzip in one command using AIX

Use tar and gzip to copy files more efficiently across network between hosts

double hash and double percent in shell variables to trim off characters from variables

The best shell script ever. An example of how scripting should be done

Use Input Field Separator (IFS) to split a set of numbers into its constituent parts

Split a file into pieces

Strip out duplicate lines with this clever awk trick

split string (eg. filename) into separate variables using set

Centre align text on a line in bash shell

Right justify text (or repeat characters) on a line with leader dots!

cron jobs not submitted - daemon run out of child processes?

Find long-running processes with a cron job

Process command line arguments in shell

Carry XWindows settings across sessions

Cross-platform version of whoami

Set terminal title from command line

Remove blank lines and comments (also indented ones) from a file

Return elements of an array in Korn shell

A decent Unix Prompt

Simple arithmetic

Script encryption and passphrase protection

Virtual host configuration in Apache http.conf

Mount a website (or any other remote resource) locally using WebDav

Using Borg to backup to a remote server via sshfs

Use sshfs to mount a remote filesystem locally

Add current hostname to list of hosts on an xcat server

What is todays 'Day Of the Year' number?

Convert Julian day numbers to dates

Send crontab job output to a date formatted log file

Edit crontab file without crontab -e

Use shell to convert a number in scientific notation to normal

Check for jobs running longer that 24 hours

Global search and replace with perl (restricted to list of files provided by Unix find command)