| Next revision | Previous revision |
| unix_linux [2018/12/06 21:05] – created 91.177.234.129 | unix_linux [2025/12/09 15:22] (current) – external edit 127.0.0.1 |
|---|
| ====== Unix/Linux ====== | * [[https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html|bash reference manual - gnu.org]] |
| | * [[http://www.kornshell.com/doc/faq.html|Lots of interesting facts about Korn shell ksh93 (including a shell compiler shcomp) - kornshell.com]] |
| | * [[http://mywiki.wooledge.org/BashPitfalls#for_f_in_.24.28ls_.2A.mp3.29|Bash Pitfalls]] - like why does this for f in $(ls *.mp3) not work?!! |
| | * [[http://hyperpolyglot.org/unix-shells#var-expansion|Good shell comparison including variable expansion - hyperpolyglot.org]] |
| | * [[http://pubs.opengroup.org/onlinepubs/9699919799/utilities/toc.html|Shell coding standards - opengroup.org]] |
| | * [[http://aix4admins.blogspot.be|aix4admins]] |
| | * [[http://linuxgazette.net/issue55/okopnik.html|The Deep, Dark Secrets of Bash]] |
| | * [[http://www.speakeasy.org/~arkay/216-7.4KshFunctions.html|Korn Shell Functions]] |
| | * [[http://www.grymoire.com/Unix/AwkRef.html|AWK reference from Bruce Barnett]] |
| | * [[http://www.catonmat.net/blog/wp-content/uploads/2008/09/awk1line.txt|HANDY AWK ONE-LINERS]] |
| | * [[http://www.grymoire.com/unix/Quote.html|Unix quoting from from Bruce Barnett (grymoire)]] |
| | * [[http://www.mpi-inf.mpg.de/departments/rg1/teaching/unixffb-ss98/quoting-guide.html|Another excellent quoting tutorial]] |
| | * [[http://aix4admins.blogspot.be/2015/02/snapshot-fs-snapshot-snapshot-is-state.html|AIX snapshot technology]] |
| | * [[RaspberryPi]] |
| | * [[SSH]] |
| | * [[http://www.anattatechnologies.com/q/2013/04/multiuser-screen/|Screen sharing from Unix using screen]] |
| | * [[https://robots.thoughtbot.com/a-tmux-crash-course|Screen sharing from Unix using tmux]] |
| | * [[http://www.csb.yale.edu/userguides/wordprocess/vi-summary.html|Handy ex and vi reference]] |
| | * [[Fedora install for my own purposes]] |
| | * [[http://www-01.ibm.com/support/docview.wss?uid=swg21282999|UNIX commands that can be used for performance-related issues]] |
| |
| * [[http://hyperpolyglot.org/unix-shells#var-expansion|Good shell comparison including variable expansion - hyperpolyglot.org]] | * [[http://www.billharlan.com/papers/Bourne_shell_idioms.html|Bourne shell idioms - billharlan.com]] |
| * [[http://pubs.opengroup.org/onlinepubs/9699919799/utilities/toc.html|Shell coding standards - opengroup.org]] | |
| * [[http://aix4admins.blogspot.be|aix4admins]] | |
| * [[http://linuxgazette.net/issue55/okopnik.html|The Deep, Dark Secrets of Bash]] | |
| * [[http://www.speakeasy.org/~arkay/216-7.4KshFunctions.html|Korn Shell Functions]] | |
| * [[http://www.grymoire.com/Unix/AwkRef.html|AWK reference from Bruce Barnett]] | |
| * [[http://www.catonmat.net/blog/wp-content/uploads/2008/09/awk1line.txt|HANDY AWK ONE-LINERS]] | |
| * [[http://www.grymoire.com/unix/Quote.html|Unix quoting from from Bruce Barnett (grymoire)]] | |
| * [[http://www.mpi-inf.mpg.de/departments/rg1/teaching/unixffb-ss98/quoting-guide.html|Another excellent quoting tutorial]] | |
| * [[http://aix4admins.blogspot.be/2015/02/snapshot-fs-snapshot-snapshot-is-state.html|AIX snapshot technology]] | |
| * [[RaspberryPi]] | |
| * [[SSH]] | |
| * [[http://www.anattatechnologies.com/q/2013/04/multiuser-screen/|Screen sharing from Unix using screen]] | |
| * [[https://robots.thoughtbot.com/a-tmux-crash-course|Screen sharing from Unix using tmux]] | |
| * [[http://www.csb.yale.edu/userguides/wordprocess/vi-summary.html|Handy ex and vi reference]] | |
| * [[Fedora install for my own purposes]] | |
| |
| =====How to automatically download and process email attachments from Gmail===== | |
| ==== Useful references==== | ==== Find all database-related files from the controlfile ==== |
| * [[http://www.courier-mta.org/maildrop/maildrop.html|maildrop]] | Assumes a backup to trace of the controlfile is available |
| * [[http://www.wonkity.com/~wblock/docs/html/maildrop.html|Using maildrop As A Mail Delivery Agent (MDA)]] | <code> |
| * [[http://manpages.ubuntu.com/manpages/trusty/man7/maildropex.7.html|.mailfilter examples]] | perl -ne 'print unless 1../Set #2. RESETLOGS/' "${CTRLFILE}"|grep -v SETCONFIG|grep -E "\'.+\/"|awk -F"'" '{print "rm -f "$2}' |
| * [[https://www.axllent.org/docs/view/gmail-pop3-with-fetchmail/|Setting up fetchmail with Gmail]] | </code> |
| * [[https://www.linode.com/docs/email/clients/using-fetchmail-to-retrieve-email/|Using Fetchmail to Retrieve Email]] | |
| | ==== List files using find without recursively descending into subdirectories ==== |
| | * [[https://unix.stackexchange.com/questions/786454/depth-1-vs-mindepth-1-maxdepth-1|Stack Exchange]] |
| | The idiomatic way is |
| | <code> |
| | find . ! -name . -prune -name "filestosearch*" |
| | </code> |
| | |
| | The key is the placement on the line of the -prune parameter |
| | <code> |
| | set lines 1000 |
| | col stmt for a500 |
| | select 'find '||destination||' -mtime +28 -prune -type f -ls' stmt from v$archive_dest where destination is not null; |
| | select 'find '||destination||' -mtime +28 -prune -type f -exec rm {} \;' stmt from v$archive_dest where destination is not null; |
| | </code> |
| | This searches for and deletes all files in the archivelog destination older than 28 days without descending into the FRA subdirectory! It does no harm as the backup retention is only guaranteed for 28 days anyway. So all this does is clean up files that may have been lying around after a clone startup or before a db was switched to noarchivelog mode. |
| | |
| | ==== Print shell commands as they are being run ==== |
| | * [[https://stackoverflow.com/questions/71117953/how-to-write-bash-function-to-print-and-run-command-when-the-command-has-argumen|tackoverflow]] |
| | Note: Support for local - was added in Bash 4.4. |
| | <code> |
| | function print_and_run_cmd |
| | { |
| | local PS4='Running cmd: ' |
| | local - |
| | set -o xtrace |
| | |
| | "$@" |
| | } |
| | </code> |
| | |
| | ==== How to assign a certain keyboard shortcut to paste specific item ==== |
| | If installed, xsel and xvkbd can be used |
| | <code> |
| | xsel -ib <<< 'Your string goes here' |
| | |
| | xsel -ib < your-file.txt |
| | |
| | xvkbd -file - <<< 'Your string goes here' |
| | |
| | xvkbd -file your-file.txt |
| | |
| | </code> |
| | Assigning it to a Key combination in the window manager might require enclosing everything in quotes |
| | <code> |
| | bash -c "xsel -ib <<< 'my string'" |
| | |
| | </code> |
| | |
| | ==== A poor man's 'expect' ==== |
| | * [[https://www.shellscript.sh/hints.html|Logging into, and using, telnet in batch mode, having the commands piped to it from a shell script]] |
| | Or, using a shell's own logfile to interrogate what it is, itself, doing! |
| | |
| | Use something like this |
| | <code> |
| | ./telnet_prompts.ksh | telnet >telnet.log |
| | </code |
| | This runs the shell and pipes its output to telnet. All the output is sent to a logfile. This logfile will be read by the shell script! |
| | |
| | telnet_prompts.ksh looks like this |
| | <code> |
| | #!ksh |
| | host="127.0.0.1" |
| | port=23 |
| | login="someuser" |
| | passwd="somepass" |
| | cmd="ls" |
| | timeout=3 |
| | file="telnet.log" |
| | prompt="$" |
| | |
| | echo open ${host} ${port} |
| | sleep 1 |
| | tout=${timeout} |
| | while [ "${tout}" -ge 0 ] |
| | do |
| | if tail -1 "${file}" 2>/dev/null | egrep -e "login:" > /dev/null |
| | then |
| | echo "${login}" |
| | sleep 1 |
| | tout=-5 |
| | continue |
| | else |
| | sleep 1 |
| | tout=`expr ${tout} - 1` |
| | fi |
| | done |
| | |
| | if [ "${tout}" -ne "-5" ]; then |
| | exit 1 |
| | fi |
| | |
| | tout=${timeout} |
| | while [ "${tout}" -ge 0 ] |
| | do |
| | if tail -1 "${file}" 2>/dev/null | egrep -e "Password:" > /dev/null |
| | then |
| | echo "${passwd}" |
| | sleep 1 |
| | tout=-5 |
| | continue |
| | else |
| | if tail -1 "${file}" 2>/dev/null | egrep -e "${prompt}" > /dev/null |
| | then |
| | tout=-5 |
| | else |
| | sleep 1 |
| | tout=`expr ${tout} - 1` |
| | fi |
| | fi |
| | done |
| | |
| | if [ "${tout}" -ne "-5" ]; then |
| | exit 1 |
| | fi |
| | |
| | > ${file} |
| | |
| | echo ${cmd} |
| | sleep 1 |
| | echo exit |
| | </code> |
| | |
| | |
| | |
| | ==== Remove a directory from the PATH variable ==== |
| | Change all the colons into carriage returns, exclude the lines containing the directory and rejoin the lines with a colon! |
| | <code> |
| | export PATH=$(echo $PATH | tr ":" "\n" | grep -v '/usr/local/bin' | xargs | tr ' ' ':') |
| | </code> |
| | ==== Random password generator for SQL*Plus or PL/SQL ==== |
| | <code> |
| | select dbms_random.string('a',15)||'#'||dbms_random.string('x',16) from dual; |
| | </code> |
| | |
| | ==== Inventory commands for AIX ==== |
| | From https://www.netiq.com/documentation/platespin-recon-42/recon_user/data/b17df8u6.html |
| | <code> |
| | Command Description |
| | rpm Get a list of installed programs. |
| | ps Get service (daemon) information. |
| | mount Get list of mounted volumes. |
| | df Get size information for mounted volumes. |
| | lslv Get SerialNumber and Label for mounted volumes. |
| | hostname Get hostname. |
| | namerslv Get DNS domain name. |
| | getconf Determine whether a 64-bit kernel. |
| | uname Determine OS release version. |
| | ifconfig Get configured network adapters and their IP address. |
| | netstat Get MAC address. |
| | entstat Get speed of network adapter. |
| | prtconf Get processor information and default gateway. |
| | lspv Get list of disks and their partitions. |
| | bootinfo Get disk size. |
| | lscfg Get disk model, manufacturer and type and network adapter model. |
| | lsitab Get daemon information. |
| | getconf Get machine memory and bootloader information. |
| | lsattr Get machine serial number and model,hyperthreading details and subnet mask. |
| | </code> |
| | ==== Monitoring commands for AIX ==== |
| | <code> |
| | Command Description |
| | vmstat Get Available Memory in Bytes |
| | ifconfig Get NIC information |
| | entstat Get Network Packets, Bytes Send/Received |
| | mpstat Get Percentage processor time |
| | iostat Get Disk transfer per second, disk idle, disk byte per second |
| | sar Get Disk Queue length, Percentage processor used |
| | lspv Get Disk partitioned or Unpartitioned space, disk list |
| | bootinfo Get disk size |
| | df Get the volume level information |
| | oslevel Get the version of AIX |
| | </code> |
| | |
| | ==== AIX hacmp (PowerHA) startup and shutdown scripts ==== |
| | <code> |
| | /usr/es/sbin/cluster/utilities/cllsserv -h -c |
| | </code> |
| | gives something like |
| | <code> |
| | #Name:Start_script:Stop_script:start_mode:cpu_usage_monitor:process_to_monitor_cpu:cpu_usage_monitor_interval: |
| | hn481_app:/ux/hacmp/bin/app_start.ksh HN481:/ux/hacmp/bin/app_stop.ksh HN481:background:no::10: |
| | </code> |
| | |
| | === The app_start.sh script referenced above === |
| | <code> |
| | #!/usr/bin/ksh |
| | #----------------------------------------------------------------- |
| | # |
| | # start_applications |
| | # |
| | #----------------------------------------------------------------- |
| | # |
| | # Customer : CLN |
| | # Environment : ALL |
| | # version : 3.0 |
| | # date : 11/2008 |
| | # syntax : app_start.ksh |
| | # description : This script is called by HACMP to start the HA |
| | # applications. It reads ../hacmp/rc2.d directory |
| | # and runs all script beginning whith S* |
| | # remarks : |
| | # history : |
| | # |
| | #================================================================== |
| | FILENAME=`basename $0` |
| | |
| | if [ $# -ne 1 ] |
| | then |
| | echo "$FILENAME require a parameter" |
| | exit 1 |
| | fi |
| | |
| | NODE=$1 |
| | |
| | if [ ! -d /ux/hacmp/$NODE ] |
| | then |
| | echo "!!! No /ux/hacmp/$NODE directory available !!!" |
| | exit 1 |
| | fi |
| | |
| | export HACMP_RG_NAME=$NODE |
| | |
| | APPBIN=/ux/hacmp/$NODE/bin |
| | APPLOG=/uxlog/hacmp/$NODE |
| | APPFIL=/ux/hacmp/$NODE/file |
| | APPTMP=/ux/hacmp/$NODE/tmp |
| | APPRC2=/ux/hacmp/$NODE/rc2.d |
| | |
| | if [ ! -d $APPLOG ] |
| | then |
| | for dir in app oracle samba |
| | do |
| | mkdir -p $APPLOG/$dir |
| | done |
| | fi |
| | |
| | LOGFILE="$APPLOG/$FILENAME.`date +%d%m%y`.`date +%H%M`.log" |
| | exec 1>$LOGFILE 2>&1 |
| | |
| | |
| | DATE=`date '+%Y-%m-%d_%H:%M:%S'` |
| | echo '\n###' |
| | echo "### Start of $0 - $DATE" |
| | |
| | for f in $APPRC2/S* |
| | { |
| | if [ -s ${f} ] |
| | then |
| | DATE=`date '+%Y-%m-%d_%H:%M:%S'` |
| | echo '\n###' |
| | echo "### Start of $f - $DATE" |
| | echo '###\n' |
| | |
| | /usr/bin/ksh ${f} start |
| | rc=$? |
| | |
| | DATE=`date '+%Y-%m-%d_%H:%M:%S'` |
| | echo '\n###' |
| | echo "### End of $f - RC=$rc - $DATE" |
| | echo '###\n' |
| | fi |
| | } |
| | |
| | DATE=`date '+%Y-%m-%d_%H:%M:%S'` |
| | echo '\n###' |
| | echo "### End of $0 - $DATE" |
| | |
| | |
| | exec 1<&- |
| | |
| | exit 0 |
| | |
| | </code> |
| | |
| | |
| | ==== Other userful cluster commands ==== |
| | <code> |
| | /usr/es/sbin/cluster/utilities/cllsres |
| | </code> |
| | <code> |
| | APPLICATIONS="hn481_app" |
| | FILESYSTEM="" |
| | FORCED_VARYON="false" |
| | FSCHECK_TOOL="fsck" |
| | FS_BEFORE_IPADDR="true" |
| | RECOVERY_METHOD="sequential" |
| | SERVICE_LABEL="hn481" |
| | SSA_DISK_FENCING="false" |
| | VG_AUTO_IMPORT="false" |
| | VOLUME_GROUP="ora1dwhtvg ora1prgvg app1prgvg app1tstvg ora1tstvg ora1tstarchvg hn481vg" |
| | USERDEFINED_RESOURCES="" |
| | </code> |
| | |
| | <code> |
| | /usr/es/sbin/cluster/events/cmd/cllsev -h -c |
| | </code> |
| | <code> |
| | #name:cmd:desc |
| | acquire_service_addr:/usr/es/sbin/cluster/events/acquire_service_addr:Script run when a service address is acquired on a node. |
| | acquire_takeover_addr:/usr/es/sbin/cluster/events/acquire_takeover_addr:Script run when a service address is acquired on a node. |
| | admin_op:/usr/es/sbin/cluster/events/admin_op:Script run when an administrator initiates a cluster operation. |
| | cluster_ffdc:/usr/es/sbin/cluster/events/cl_ffdc:Script run to collect first failure data capture |
| | cluster_notify:/usr/es/sbin/cluster/events/cluster_notify:Script run when SystemMirror detects a cluster configuration problem |
| | config_too_long:/usr/es/sbin/cluster/events/config_too_long:Script run when a cluster event has been running for a long time. |
| | event_error:/usr/es/sbin/cluster/events/event_error:Script run when there is a unrecoverable error while running a cluster event. |
| | external_resource_state_change:/usr/es/sbin/cluster/events/external_resource_state_change:Scripts run when a cluster resource encounters a recoverable error |
| | external_resource_state_change_complete:/usr/es/sbin/cluster/events/external_resource_state_change_complete:Scripts run when a cluster resource encounters a recoverable error |
| | fail_interface:/usr/es/sbin/cluster/events/fail_interface:Script run after a network interface has failed. |
| | forced_down_too_long:/usr/es/sbin/cluster/events/forced_down_too_long:Script run when a cluster node has been in the unmanaged state for some time |
| | get_disk_vg_fs:/usr/es/sbin/cluster/events/get_disk_vg_fs:Script run to acquire disks, varyon volume groups, and mount filesystems. |
| | intersite_fallover_prevented:/usr/es/sbin/cluster/events/intersite_fallover_prevented:Script run when a cluster resource is not moved across a site boundary |
| | join_interface:/usr/es/sbin/cluster/events/join_interface:Script run after a network interface has recovered. |
| | network_down:/usr/es/sbin/cluster/events/network_down:Script run when a network has failed. |
| | network_down_complete:/usr/es/sbin/cluster/events/network_down_complete:Script run after the network_down script has successfully completed. |
| | network_stable:/usr/es/sbin/cluster/events/network_stable:Script run when a network is stable. |
| | network_unstable:/usr/es/sbin/cluster/events/network_unstable:Script run when a network is continuously changing state. |
| | network_up:/usr/es/sbin/cluster/events/network_up:Script run after a network has become active. |
| | network_up_complete:/usr/es/sbin/cluster/events/network_up_complete:Script run after the network_up script has successfully completed. |
| | node_down:/usr/es/sbin/cluster/events/node_down:Script run when cluster services are stopped or a node has failed. |
| | node_down_complete:/usr/es/sbin/cluster/events/node_down_complete:Script run after the node_down script has successfully completed. |
| | node_down_local:/usr/es/sbin/cluster/events/node_down_local:Script run when it is the local node which is leaving the cluster. |
| | node_down_local_complete:/usr/es/sbin/cluster/events/node_down_local_complete:Script run after the node_down_local script has successfully completed. |
| | node_down_remote:/usr/es/sbin/cluster/events/node_down_remote:Script run when it is a remote node which is leaving the cluster. |
| | node_down_remote_complete:/usr/es/sbin/cluster/events/node_down_remote_complete:Script run after the node_down_remote script has successfully completed. |
| | node_up:/usr/es/sbin/cluster/events/node_up:Script run when a node is attempting to join the cluster. |
| | node_up_complete:/usr/es/sbin/cluster/events/node_up_complete:Script run after the node_up script has successfully completed. |
| | node_up_local:/usr/es/sbin/cluster/events/node_up_local:Script run when it is the local node which is joining the cluster. |
| | node_up_local_complete:/usr/es/sbin/cluster/events/node_up_local_complete:Script run after the node_up_local script has successfully completed. |
| | node_up_remote:/usr/es/sbin/cluster/events/node_up_remote:Script run when it is a remote node which is joining the cluster. |
| | node_up_remote_complete:/usr/es/sbin/cluster/events/node_up_remote_complete:Script run after the node_up_remote script has successfully completed. |
| | reconfig_configuration_complete:/usr/es/sbin/cluster/events/reconfig_configuration_complete:Script run while processing a dynamic configuration change. |
| | reconfig_resource_acquire:/usr/es/sbin/cluster/events/reconfig_resource_acquire:Script run while processing a dynamic configuration change. |
| | reconfig_resource_acquire_fence:/usr/es/sbin/cluster/events/reconfig_resource_acquire_fence:Script run while processing a dynamic configuration change. |
| | reconfig_resource_acquire_secondary:/usr/es/sbin/cluster/events/reconfig_resource_acquire_secondary:Script run while processing a dynamic configuration change. |
| | reconfig_resource_complete:/usr/es/sbin/cluster/events/reconfig_resource_complete:Script run while processing a dynamic configuration change. |
| | reconfig_resource_complete_secondary:/usr/es/sbin/cluster/events/reconfig_resource_complete_secondary:Script run while processing a dynamic configuration change. |
| | reconfig_resource_release:/usr/es/sbin/cluster/events/reconfig_resource_release:Script run while processing a dynamic configuration change. |
| | reconfig_resource_release_fence:/usr/es/sbin/cluster/events/reconfig_resource_release_fence:Script run while processing a dynamic configuration change. |
| | reconfig_resource_release_primary:/usr/es/sbin/cluster/events/reconfig_resource_release_primary:Script run while processing a dynamic configuration change. |
| | reconfig_resource_release_secondary:/usr/es/sbin/cluster/events/reconfig_resource_release_secondary:Script run while processing a dynamic configuration change. |
| | reconfig_topology_complete:/usr/es/sbin/cluster/events/reconfig_topology_complete:Script run while processing a dynamic configuration change. |
| | reconfig_topology_start:/usr/es/sbin/cluster/events/reconfig_topology_start:Script run while processing a dynamic configuration change. |
| | release_service_addr:/usr/es/sbin/cluster/events/release_service_addr:Script run when a service address is released from a node. |
| | release_takeover_addr:/usr/es/sbin/cluster/events/release_takeover_addr:Script run when a service address is released from a node. |
| | release_vg_fs:/usr/es/sbin/cluster/events/release_vg_fs:Script run to unmount filesystems and varyoff volume groups. |
| | rep_disk_notify:/usr/es/sbin/cluster/events/rep_disk_notify:Script run when a repository disk has failed |
| | resource_group_in_error:/usr/es/sbin/cluster/events/resource_group_in_error:Script run when a resource group goes to error state |
| | resource_state_change:/usr/es/sbin/cluster/events/resource_state_change:Scripts run when a cluster resource encounters a recoverable error or the user moves resource groups. |
| | resource_state_change_complete:/usr/es/sbin/cluster/events/resource_state_change_complete:Scripts run when a cluster resource encounters a recoverable error or the user moves resource groups. |
| | rg_move:/usr/es/sbin/cluster/events/rg_move:Scripts run when SystemMirror moves resources and resource groups. |
| | rg_move_acquire:/usr/es/sbin/cluster/events/rg_move_acquire:Scripts run when SystemMirror moves resources and resource groups. |
| | rg_move_complete:/usr/es/sbin/cluster/events/rg_move_complete:Scripts run when SystemMirror moves resources and resource groups. |
| | rg_move_fence:/usr/es/sbin/cluster/events/rg_move_fence:Scripts run when SystemMirror moves resources and resource groups. |
| | rg_move_release:/usr/es/sbin/cluster/events/rg_move_release:Scripts run when SystemMirror moves resources and resource groups. |
| | server_down:/usr/es/sbin/cluster/events/server_down:Script run when an application has failed. |
| | server_down_complete:/usr/es/sbin/cluster/events/server_down_complete:Script run when an application has failed. |
| | server_restart:/usr/es/sbin/cluster/events/server_restart:Script run to restart an application. |
| | server_restart_complete:/usr/es/sbin/cluster/events/server_restart_complete:Script run to restart an application. |
| | site_down:/usr/es/sbin/cluster/events/site_down:Script run when all nodes in a site are down. |
| | site_down_complete:/usr/es/sbin/cluster/events/site_down_complete:Script run after the site_down script has successfully completed. |
| | site_down_local:/usr/es/sbin/cluster/events/site_down_local:Script run when it is the local site which is leaving the cluster. |
| | site_down_local_complete:/usr/es/sbin/cluster/events/site_down_local_complete:Script run after the site_down_local script has successfully completed. |
| | site_down_remote:/usr/es/sbin/cluster/events/site_down_remote:Script run when it is a remote site which is leaving the cluster. |
| | site_down_remote_complete:/usr/es/sbin/cluster/events/site_down_remote_complete:Script run after the site_down_remote script has successfully completed. |
| | site_isolation:/usr/es/sbin/cluster/events/site_isolation:Scripts run when SystemMirror detects a site isolation or merge condition. |
| | site_isolation_complete:/usr/es/sbin/cluster/events/site_isolation_complete:Script run after a site_isolation event completes. |
| | site_merge:/usr/es/sbin/cluster/events/site_merge:Scripts run when SystemMirror detects a site isolation or merge condition. |
| | site_merge_complete:/usr/es/sbin/cluster/events/site_merge_complete:Script run after a site_merge event completes. |
| | site_up:/usr/es/sbin/cluster/events/site_up:Script run when cluster services are started on the first node in a site. |
| | site_up_complete:/usr/es/sbin/cluster/events/site_up_complete:Script run after the site_up script has successfully completed. |
| | site_up_local:/usr/es/sbin/cluster/events/site_up_local:Script run when it is the local site which is joining the cluster. |
| | site_up_local_complete:/usr/es/sbin/cluster/events/site_up_local_complete:Script run after the site_up_local script has successfully completed. |
| | site_up_remote:/usr/es/sbin/cluster/events/site_up_remote:Script run when it is a remote site which is joining the cluster. |
| | site_up_remote_complete:/usr/es/sbin/cluster/events/site_up_remote_complete:Script run after the site_up_remote script has successfully completed. |
| | split_merge_prompt:/usr/es/sbin/cluster/utilities/cl_sm_prompt:Script to prompt the operator for manual choice on split or merge |
| | start_server:/usr/es/sbin/cluster/events/start_server:Script run to start an application. |
| | start_udresource:/usr/es/sbin/cluster/events/start_udresource:Script run to start application servers. |
| | stop_server:/usr/es/sbin/cluster/events/stop_server:Script run to start an application. |
| | stop_udresource:/usr/es/sbin/cluster/events/stop_udresource:Script run to stop application servers. |
| | swap_adapter:/usr/es/sbin/cluster/events/swap_adapter:Script run to swap IP Addresses between two network interfaces. |
| | swap_adapter_complete:/usr/es/sbin/cluster/events/swap_adapter_complete:Script run after the swap_adapter script has successfully completed |
| | </code> |
| | |
| | ==== cat a local file to the end of a file on a remote host over ssh ==== |
| | Nice trick but not really a trick! It saves having to scp the file over and then login and cat it on the remote host. |
| | |
| | The contents of the local file are catted to STDOUT which is piped over ssh to a waiting cat which appends the incoming STDIN to the specified file. |
| | <code> |
| | cat id_rsa.pub | ssh oracle@hn5306 'cat >> .ssh/authorized_keys' |
| | </code> |
| | |
| | ==== Countdown timer loop ==== |
| | Two examples from /usr/es/sbin/cluster/etc/rc.cluster |
| | <code> |
| | #!/bin/ksh93 |
| | |
| | typeset i=3 # make sure init has time to start it |
| | |
| | while (( i-- )) |
| | do |
| | src_running=$(ps -e | awk '$NF == "srcmstr" { print $1; exit }') |
| | |
| | [[ -n "$src_running" ]] && break |
| | done |
| | |
| | if (( $i == 0 )) |
| | then |
| | cl_echo 234 "\n$PROGNAME: srcmstr is not running.\n" $PROGNAME |
| | exit 1 |
| | fi |
| | </code> |
| | <code> |
| | if [[ -n "$src_running" ]] |
| | then |
| | cl_echo 235 "Checking for srcmstr active...\n" |
| | typeset -i i=10 # try ten times to contact it |
| | |
| | while (( i-- )) |
| | do |
| | lssrc -s inetd >/dev/null 2>&1 && break # break out on success |
| | sleep 1 # otherwise wait a second and try again |
| | echo ".\c" |
| | done |
| | |
| | if (( i == 0 )) |
| | then |
| | cl_echo 236 "\n\nERROR: srcmstr is not accepting connections\n" |
| | exit 1 |
| | fi |
| | |
| | cl_echo 237 "complete.\n" |
| | fi |
| | </code> |
| | |
| | ==== Sort a comma separated list in shell ==== |
| | * [[https://stackoverflow.com/questions/47665195/how-to-sort-comma-separated-values-in-bash|How to sort comma separated values in shell]] |
| | Not as easy as it sounds. Googling it shows many varied ways mostly using ugly label syntax.\\ |
| | In amongst all the chaff, I spotted this wheat! |
| | <code> |
| | echo "7, 15, 6, 2, -9" | sed -e $'s/,/\n/g' | sort -n | tr '\n' ',' | sed 's/,$//' |
| | sed -e $'s/,/\n/g' : For splitting string into lines by comma. |
| | sort -n : Then you can use sort by number |
| | tr '\n' ',' : Convert newline separator back to comma. |
| | sed 's/,$//' : Removing trailing comma. |
| | </code> |
| | It's that dollar sign that makes it possible. Without it, it fails. I have no idea what it does and I can find no info on it! |
| | |
| | ==== Generate a random sequence of characters (maybe for a password) ==== |
| | <code> |
| | echo $(< /dev/urandom tr -dc %A-Z-a-z-0-9 | head -c${1:-30}) |
| | </code> |
| | As that one might generate a number as the first character, this one generates 7 characters, followed by a '#', followed by 8 numbers. Should satisfy most |
| | <code> |
| | ORAPW=$(echo "$(tr -dc A-Za-z</dev/urandom|head -c7)#$(tr -dc 0-9</dev/urandom|head -c8)") |
| | </code> |
| | |
| | |
| | ==== How to automatically download and process email attachments from Gmail ==== |
| | === Useful references === |
| | * [[http://www.courier-mta.org/maildrop/maildrop.html|maildrop]] |
| | * [[http://www.wonkity.com/~wblock/docs/html/maildrop.html|Using maildrop As A Mail Delivery Agent (MDA)]] |
| | * [[http://manpages.ubuntu.com/manpages/trusty/man7/maildropex.7.html|.mailfilter examples]] |
| | * [[https://www.axllent.org/docs/view/gmail-pop3-with-fetchmail/|Setting up fetchmail with Gmail]] |
| | * [[https://revadig.blogspot.com/2017/08/fetchmail-check-new-email-in-gmail.html|Fetchmail, Check new email in gmail, yahoo and other accounts]] |
| | * [[https://www.howtoforge.com/debian_etch_fetchmail|Retrieving Emails From Remote Servers With fetchmail (Debian Etch)]] |
| | * [[https://www.linode.com/docs/email/clients/using-fetchmail-to-retrieve-email/|Using Fetchmail to Retrieve Email]] |
| | * [[http://www.unixmantra.com/2013/04/aix-powerha-hacmp-commands.html|Useful PowerHA (hacmp) commands]] |
| |
| Eventually, after months of trying different things, came up with this combination of tools that worked very well. | Eventually, after months of trying different things, came up with this combination of tools that worked very well. |
| ====Install tools==== | === Install tools === |
| <code>0@@</code> | <code> |
| * Set Gmail account up for POP and disable IMAP | sudo dnf -y install fetchmail maildrop uudeview openssl |
| ====Configure fetchmail==== | </code> |
| | * Set Gmail account up for POP and disable IMAP |
| | === Configure fetchmail === |
| This is the part that talks to Gmail and downloads the emails to the local machine. | This is the part that talks to Gmail and downloads the emails to the local machine. |
| <code>1@@</code> | <code> |
| | vi ~/.fetchmailrc |
| | |
| | set postmaster "<username>" |
| | #set daemon 600 |
| | poll pop.gmail.com with proto POP3 |
| | user "account@gmail.com" there with password '<password>' is <username> here options ssl nokeep |
| | mda "/usr/bin/maildrop .mailfilter" |
| | # sslcertfile /etc/ssl/certs/ca-bundle.crt |
| | </code> |
| If fetchmail complains about ssl certificates, try specifying wherein is with the sslcertfile line | If fetchmail complains about ssl certificates, try specifying wherein is with the sslcertfile line |
| ==== Configure maildrop==== | === Configure maildrop === |
| <code>2@@</code> | <code> |
| | maildirmake maildrop |
| | maildirmake backup |
| | |
| | vi ~/.mailfilter |
| | |
| | LOGFILE = "/home/dbahawk/maildrop/maildrop.log" |
| | DEFAULT="$HOME/maildrop" |
| | |
| | # keep a copy |
| | cc backup |
| | `cd backup/new && rm -f dummy \`ls -t | sed -e 1,50d\`` |
| | |
| | if (/^To:.*getmail@...\.dnsalias\.org$/) |
| | { |
| | to Maildir/getmail/ |
| | } |
| | |
| | #if (/^To:.*dbahawk/) |
| | # { |
| | # to maildrop |
| | # } |
| | |
| | if (/^To:.*dbahawk/) |
| | dotlock "auto.lock" { |
| | to "|uudeview -c -i" |
| | } |
| | </code> |
| Not really sure why this filter is not working properly. The cc works but the To: check doesn't. So I have a shell that runs separately to process the emails in the backup mail directory. | Not really sure why this filter is not working properly. The cc works but the To: check doesn't. So I have a shell that runs separately to process the emails in the backup mail directory. |
| |
| ====uudeview==== | === uudeview === |
| This is a clever program that can extract attachments from emails and put them in a directory of you choosing.<br /> | This is a clever program that can extract attachments from emails and put them in a directory of you choosing.\\ |
| This shell runs from cron at regular intervals to run the whole process | This shell runs from cron at regular intervals to run the whole process |
| <code>3@@</code> | <code> |
| | #!/usr/bin/ksh |
| |
| =====Use sed to repeat characters===== | # fetch any new mail from Gmail (uses .fetchmailrc) |
| | # fetchmail log is $HOME/maildrop/maildrop.log |
| | fetchmail -d0 |
| | |
| | # fetchmail uses maildrop as mda (uses .mailfilter) to save messages to a maildir. |
| | # this is not working correctly but still copies the messages to backup |
| | |
| | # pull out the attachments and send them to the incoming directory for the collector to process when it wakes up |
| | uudeview -c -i -p ~/Downloads/dbahawk_forward backup/new/* |
| | </code> |
| | |
| | ==== Reset terminal to a sane state if it has got messed up ==== |
| | If, for example, you have done a cat on an binary file, sometines the terminal gets screwed up. This can get it back. |
| | <code> |
| | <CTRL-J>stty sane<CTRL-J> |
| | </code> |
| | The use of CTRL + J (Linefeed) is in case the Return key has also stopped working. |
| | |
| | ==== The underused xargs command! ==== |
| | See man xargs |
| | <code> |
| | ls | xargs -t -I {} mv {} {}.old This command sequence renames all files in the current directory by adding .old to the end of each name. The -I flag tells the xargs |
| | command to insert each line of the ls directory listing where {} (braces) appear. If the current directory contains the files chap1, chap2, and chap3, this constructs |
| | the following commands: |
| | |
| | mv chap1 chap1.old |
| | mv chap2 chap2.old |
| | mv chap3 chap3.old |
| | </code> |
| | |
| | ==== Watch a directory and automatically perform an action when new files come into it ==== |
| | Using a watchdog program like this Python script can do the trick ([[https://www.michaelcho.me/article/using-pythons-watchdog-to-monitor-changes-to-a-directory|Using Python's Watchdog to monitor changes to a directory]]) |
| | <code> |
| | import time |
| | from watchdog.observers import Observer |
| | from watchdog.events import FileSystemEventHandler |
| | |
| | |
| | class Watcher: |
| | DIRECTORY_TO_WATCH = "/path/to/my/directory" |
| | |
| | def __init__(self): |
| | self.observer = Observer() |
| | |
| | def run(self): |
| | event_handler = Handler() |
| | self.observer.schedule(event_handler, self.DIRECTORY_TO_WATCH, recursive=True) |
| | self.observer.start() |
| | try: |
| | while True: |
| | time.sleep(5) |
| | except: |
| | self.observer.stop() |
| | print "Error" |
| | |
| | self.observer.join() |
| | |
| | |
| | class Handler(FileSystemEventHandler): |
| | |
| | @staticmethod |
| | def on_any_event(event): |
| | if event.is_directory: |
| | return None |
| | |
| | elif event.event_type == 'created': |
| | # Take any action here when a file is first created. |
| | print "Received created event - %s." % event.src_path |
| | |
| | elif event.event_type == 'modified': |
| | # Taken any action here when a file is modified. |
| | print "Received modified event - %s." % event.src_path |
| | |
| | |
| | if __name__ == '__main__': |
| | w = Watcher() |
| | w.run() |
| | </code> |
| | |
| | ==== Top 10 (or 20) thread count consumers in AIX ==== |
| | <code> |
| | ps -ef -o thcount,user,pid,ppid,time,args|sort -nr|head -20 |
| | </code> |
| | ==== How much memory is being used on AIX? ==== |
| | * [[https://www.ibm.com/support/pages/aix-memory-usage-or-who-using-memory-and-how/|AIX Memory Usage - or - Who is using the memory and how?]] |
| | * [[https://www.ibm.com/support/pages/aix-memory-usage-100-used-filecache-and-paging|AIX Memory usage: 100% used, filecache and paging]] |
| | * [[https://developer.ibm.com/components/aix/articles/using-svmon-to-display-available-memory-on-aix/|Using svmon to display available memory on IBM AIX]] |
| | <code> |
| | svmon -O summary=basic,unit=GB |
| | </code> |
| | <code> |
| | Unit: GB |
| | -------------------------------------------------------------------------------------- |
| | size inuse free pin virtual available mmode |
| | memory 270.00 266.92 3.08 22.9 204.16 57.2 Ded |
| | pg space 64.0 0.84 |
| | |
| | work pers clnt other |
| | pin 14.7 0 0.04 8.18 |
| | in use 204.16 0 62.8 |
| | </code> |
| | ==== Top memory consumers in AIX ==== |
| | <code> |
| | svmon -U -t 10 -O process=on -O sortentity=pgsp |
| | </code> |
| | ==== Top 10 memory-consuming processes ==== |
| | <code> |
| | ps aux | head -1; ps aux | sort -rn +3 | head |
| | </code> |
| | or, in order of real memory usage |
| | <code> |
| | ps vx | head -1; ps vx | grep -v PID | sort -rn +6 | head -10 |
| | </code> |
| | |
| | ==== Show paged memory hogs on AIX ==== |
| | Show top 10 processes using memory |
| | <code> |
| | svmon -Pt20 | perl -ne 'print if($.==2||$&&&!$s++);$.=0 if(/^-+$/)' |
| | </code> |
| | |
| | ==== Top 10 processes in order of I/O usage ==== |
| | <code> |
| | ps vx | head -1; ps vx | grep -v PID | sort -rn +4 | head -10 |
| | </code> |
| | |
| | ==== Top 10 CPU consuming processes ==== |
| | Info on High(100%) used cpu myth! [[http://www.dba-oracle.com/t_high_cpu.htm|"Chicken Little" myth - dba-oracle.com]] |
| | On AIX |
| | <code> |
| | ps aux | head -1; ps aux | sort -rn +2 | head -10 |
| | </code> |
| | then |
| | <code> |
| | ps aeww <process_id> |
| | </code> |
| | or on Linux |
| | <code> |
| | top -b -n 1 | sed -e "1,6d" | head -11 |
| | </code> |
| | gives something like... |
| | <code> |
| | PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND |
| | 29185 oracle 20 0 6597120 2.2g 74936 S 6.2 7.0 11:18.92 java |
| | 76486 oracle 20 0 129824 2168 1744 S 6.2 0.0 566:09.68 genorapw |
| | 1 root 20 0 191692 4012 2112 S 0.0 0.0 29:21.87 systemd |
| | 2 root 20 0 0 0 0 S 0.0 0.0 0:00.83 kthreadd |
| | 4 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/0:0H |
| | 6 root 20 0 0 0 0 S 0.0 0.0 24:24.49 ksoftirqd/0 |
| | 7 root rt 0 0 0 0 S 0.0 0.0 52:02.11 migration/0 |
| | 8 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcu_bh |
| | 9 root 20 0 0 0 0 S 0.0 0.0 94:06.86 rcu_sched |
| | 10 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 lru-add-drain |
| | </code> |
| | Now we have the process 12385 showing 119% of cpu usage! |
| | <code> |
| | oracle 12385 12327 99 Aug17 ? 74-09:39:08 /cln/exp/ora_bin1/app/oracle/middleware/oracle_common/jdk/bin/java -server -Xms256M -Xmx1740M -XX:PermSize=128M -XX:MaxPermSize=768M -XX:CompileThreshold=8000 -XX:-DoEscapeAnalysis -XX:+UseCodeCacheFlushing -XX:ReservedCodeCacheSize=100M -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSClassUnloadingEnabled -Dweblogic.Name=EMGC_OMS1 -Djava.security.policy=/cln/exp/ora_bin1/app/oracle/middleware/wlserver/server/lib/weblogic.policy -Dweblogic.ProductionModeEnabled=true -Dweblogic.system.BootIdentityFile=/cln/exp/ora_bin1/app/oracle/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/data/nodemanager/boot.properties -Dweblogic.nodemanager.ServiceEnabled=true -Dweblogic.nmservice.RotationEnabled=true -Dweblogic.security.SSL.ignoreHostnameVerification=true -Dweblogic.ReverseDNSAllowed=false -DINSTANCE_HOME=/cln/exp/ora_bin1/app/oracle/gc_inst/em/EMGC_OMS1 -DORACLE_HOME=/cln/exp/ora_bin1/app/oracle/middleware -Ddomain.home=/cln/exp/ora_bin1/app/oracle/gc_inst/user_projects/domains/GCDomain -Djava.awt.headless=true -Ddomain.name=GCDomain -Doracle.sysman.util.logging.mode=dual_mode -Djbo.doconnectionpooling=true -Djbo.txn.disconnect_level=1 -Docm.repeater.home=/cln/exp/ora_bin1/app/oracle/middleware -Djbo.ampool.minavailablesize=1 -Djbo.ampool.timetolive=-1 -Djbo.load.components.lazily=true -Djbo.max.cursors=5 -Djbo.recyclethreshold=50 -Djbo.ampool.maxavailablesize=50 -Djavax.xml.bind.JAXBContext=com.sun.xml.bind.v2.ContextFactory -Djava.security.egd=file:///dev/./urandom -Dweblogic.debug.DebugWebAppSecurity=true -Dweb ogic.SSL.LoginTimeoutMillis=300000 -Djps.auth.debug=true -Djps.authz=ACC -Djps.combiner.optimize.lazyeval=true -Djps.combiner.optimize=true -Djps.subject.cache.key=5 -Djps.subject.cache.ttl=600000 -Doracle.apm.home=/cln/exp/ora_bin1/app/oracle/middleware/apm/ -DAPM_HELP_FILENAME=oesohwconfig.xml -Dweblogic.data.canTransferAnyFile=true -Dhttps.protocols=TLSv1,TLSv1.1,TLSv1.2 -DHTTPClient.retryNonIdempotentRequest=false -Dweblogic.security.SSL.minimumProtocolVersion=TLSv1 -Djava.endorsed.dirs=/cln/exp/ora_bin1/app/oracle/middleware/oracle_common/jdk/jre/lib/endorsed:/cln/exp/ora_bin1/app/oracle/middleware/oracle_common/modules/endorsed -Djava.protocol.handler.pkgs=oracle.mds.net.protocol -Dopss.version=12.1.3 -Digf.arisidbeans.carmlloc=/cln/exp/ora_bin1/app/oracle/gc_inst/user_projects/domains/GCDomain/config/fmwconfig/carml -Digf.arisidstack.home=/cln/exp/ora_bin1/app/oracle/gc_inst/user_projects/domains/GCDomain/config/fmwconfig/arisidprovider -Doracle.security.jps.config=/cln/exp/ora_bin1/app/oracle/gc_inst/user_projects/domains/GCDomain/config/fmwconfig/jps-config.xml -Doracle.deployed.app.dir=/cln/exp/ora_bin1/app/oracle/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/tmp/_WL_user -Doracle.deployed.app.ext=/- -Dweblogic.alternateTypesDirectory=/cln/exp/ora_bin1/app/oracle/middleware/oracle_common/modules/oracle.ossoiap_12.1.3,/cln/exp/ora_bin1/app/oracle/middleware/oracle_common/modules/oracle.oamprovider_12.1.3,/cln/exp/ora_bin1/app/oracle/middleware/oracle_common/modules/oracle.jps_12.1.3 -Doracle.mds.filestore.preferred= -Dadf.version=12.1.3 -Dweblogic.jdbc.remoteEnabled=false -Dcommon.components.home=/cln/exp/ora_bin1/app/oracle/middleware/oracle_common -Djrf.version=12.1.3 -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.Jdk14Logger -Ddomain.home=/cln/exp/ora_bin1/app/oracle/gc_inst/user_projects/domains/GCDomain -Doracle.server.config.dir=/cln/exp/ora_bin1/app/oracle/gc_inst/user_projects/domains/GCDomain/config/fmwconfig/servers/EMGC_OMS1 -Doracle.domain.config.dir=/cln/exp/ora_bin1/app/oracle/gc_inst/user_projects/domains/GCDomain/config/fmwconfig -Dohs.product.home=/cln/exp/ora_bin1/app/oracle/middleware/ohs -da -Dwls.home=/cln/exp/ora_bin1/app/oracle/middleware/wlserver/server -Dweblogic.home=/cln/exp/ora_bin1/app/oracle/ |
