* [[http://www.oucs.ox.ac.uk/hfs/archive/cli/index.xml?splitLevel=-1#body.1_div.4|Some tips from Oxford University]] * [[http://thobias.org/tsm/sql/#toc147|Some useful TSM server queries]] * [[http://froebe.net/blog/2014/03/24/ibm-tivoli-storage-manager-tsm-how-to-determine-management-class-and-retention-using-dsmc/|TSM - How to determine management class and retention using dsmc]] - nice syntax highlighting! ===== Tivoli Storage Management ===== ==== Default location for files ==== Linux, AIX dsm.sys,dsm.opt /opt/tivoli/tsm/client/ba/bin dsmerror.log, dsmsched.log /opt/tivoli/tsm/client/ba/bin Solaris dsm.sys,dsm.opt /usr/tivoli/tsm/client/ba/bin dsmerror.log, dsmsched.log /var/log ==== Restart ITM Monitoring agent ==== === To see if the Oracle module is installed === export ITM_HOME=/opt/IBM/ITM $ITM_HOME/bin/cinfo -i and see which agents are running export ITM_HOME=/opt/IBM/ITM $ITM_HOME/bin/cinfo -r and see which agents are configured (or not) export ITM_HOME=/opt/IBM/ITM $ITM_HOME/bin/cinfo -o === See which instances have been setup to be monitored === export ITM_HOME=/opt/IBM/ITM $ITM_HOME/smitools/scripts/kdyedit -h /opt/IBM/ITM list export ITM_HOME=/opt/IBM/ITM $ITM_HOME/smitools/scripts/kciedit -h /opt/IBM/ITM list === Add an instance to be monitored === $ITM_HOME/smitools/scripts/kdyedit -h /opt/IBM/ITM -r itmora -t rz add -i $ORACLE_SID This one should also be done in order that rc.d scripts know to start the agent as itmora instead of root $ITM_HOME/smitools/scripts/kciedit -h /opt/IBM/ITM -r itmora -t rz add -i $ORACLE_SID ==== Restart the Oracle agents ==== To avoid issues with orphaned processes, restart is best done with a combination of stop, cleanup, start.\\ **Stop all Oracle agents** export ITM_HOME=/opt/IBM/ITM $ITM_HOME/smitools/scripts/StartAgent.sh stop rz or, the standard method... su - itmora -c "/bin/ksh -c '/opt/IBM/ITM/bin/itmcmd agent -o TIBPFW start rz >/tmp/itmcmd_TIBPFW.log 2>&1'" **Check agent processes have stopped** ps -ef | grep [[k]]rz Any remaining processes (containing a SID) should be killed.\\ It is normal to have agents for 61, ul and ux still running.\\ **Check agent RunInfo file is correct**\\ If this file is corrupted, the StartAgent.sh script does not function correctly vi $ITM_HOME/config/.ConfigData/RunInfo ...and remove any lines containing database SIDs in the 4th column\\ **Start all Oracle agents** $ITM_HOME/smitools/scripts/StartAgent.sh start rz or\\ ...using ITM standard command to restart one agent /usr/bin/su - itmora -c "/opt/IBM/ITM/bin/itmcmd agent -o EMREP stop or" Perform cleanup as above /usr/bin/su - itmora -c "/opt/IBM/ITM/bin/itmcmd agent -o EMREP start or" === Check the agent is running === ps -ef | grep [[k]]rz === Reconfigure agent === export ITM_HOME=/opt/IBM/ITM $ITM_HOME/bin/itmcmd config -A rz ==== Troubleshooting ==== === cinfo -r reports processes not running (as well as running) === eg: coupprod or 25690298 DMMPRD ...process not running coupprod or 5832894 DMMPRD ...process not running coupprod ux 39452858 root Apr23 None ...running coupprod or 43319374 itmora Apr24 DMMPRD ...running need to remove the first 2 lines... cd /opt/IBM/ITM/config/.ConfigData vi RunInfo and remove the unnecessary lines ==== Remove / uninstall / decommission an ITM agent ==== === Check agent is configured === export ITM_HOME=/opt/IBM/ITM $ITM_HOME/bin/cinfo -o rz === Stop the running agent === SID=WM9TNSBX $ITM_HOME/smitools/scripts/StartAgent.sh stop rz -o ${SID} === Delete config files for the instance === cd $ITM_HOME/config ls -al *rz_${SID}* rm -f *rz_${SID}* ls -al *rz_${SID}* ls: 0653-341 The file *rz_WM9TNSBX* does not exist. === Modify the rz environment file === vi $ITM_HOME/config/.ConfigData/krzenv === Modify the rz agent startup file === * Remove line from "start_all" section * Remove line from "stop_all" section vi /etc/rc.itm1 === Recheck if the agent is configured (SID should have gone from list) === $ITM_HOME/bin/cinfo -o rz ==== Automate modification of ORACLE_HOME in config files and restart ITM agent for each database ==== After an upgrade, the hard-coded(!!!) path to ORACLE_HOME needs modifying to use the new location export ITM_HOME=/opt/IBM/ITM cd ${ITM_HOME}/config for i in $(find . -type f -name "*_rz_*cfg"); do perl -p -i.bck -e 's/11.2.0.3/11.2.0.4/g' $i db=$(echo $i | awk -F_ '{print $NF}' | awk -F\\. '{print $1}') /usr/bin/su - itmora -c "/bin/ksh -c '${ITM_HOME}/bin/itmcmd agent -o $db stop rz'" /usr/bin/su - itmora -c "/bin/ksh -c '${ITM_HOME}/bin/itmcmd agent -o $db start rz'" done ${ITM_HOME}/bin/cinfo -r ==== Starting the TSM client ==== On Unix dsmc -se=so_u_`hostname`_orx_d_bax On Windows dsmc -optfile=c:\\oracle\\SID\\admin\\dsm.opt ==== Send a file to TSM ==== dsmc archive -server=SO_U_ubenoa31_ORX_D_BAX "/oracle/export/NSWORKP1/export_NSWORKP1_D_FULL_201301030007_01.dmp" ==== Query the archive to see if a file is available in TSM ==== dsmc query archive -server=SO_U_ubenoa31_ORX_D_BAX "/oracle/export/NSWORKP1/export_NSWORKP1_D_FULL_20130103*" Query all the files sent to the monthly node since start of FEB 2013 (using dateformat YYYY-MM-DD) dsmc q ar -date=3 -fromdate=2013-02-01 -subdir=yes -filesonly -se=SO_U_`uname -n`_ORX_M_BAX "/oracle/export/*" and on Windows... dsmc query archive -dateformat=3 -fromdate=2013-02-01 -subdir=yes -filesonly -optfile='C:\\home\\ibmtools\\etc\\oracle\\SO_I_WDCRHBP05_ORX_D_SOL.opt' Note: You will get an error with the above command as Windows ignores the dateformat on the command line!\\ To fix it, put dateformat 3 in the opt file.\\ ==== List all export jobs on this TSM node ==== cd /oracle/export dsmc q ar -subdir=yes -se=SO_U_UBENOA26_ORX_D_BAX "expdp*" ==== Retrieve file from TSM ==== dsmc retrieve -pick -server=SO_U_ubenoa31_ORX_D_BAX "/oracle/export/NSWORKP1/export_NSWORKP1_D_FULL_20130103*" This opens an interactive window listing all files that match the file query (note the wildcard)\\ Files can be toggled on and off for recovery by pressing the number associated with the file then recover with the letter o (for ok!).\\ If several versions of the same file exist, use the description field to make the distinction dsmc retr -server=SO_U_`hostname`_ORX_D_SOL -descr="*30-01-2014*" "/oracle/export/AGP/export_AGP_D_FULL_20130103*" ==== Retrieve a file from TSM and save it to a different name ==== dsmc retr -server=SO_U_`hostname`_ORX_D_SOL /oracle/export/AGP/export_AGP_D_FULL_201301031200.dmp renamedfile.dmp ==== Restore a file from TSM (from backup not archive) ==== dsmc rest /home/bey9at77/Documents/* /tmp/restore/ -su=yes -inactive -pick ==== Check backup is performed across DWDM (diagonal) to TSM server in remote location ==== Some customers want the backups of the databases running in one data centre to be sent to the other data centre providing "data centre down" recovery.\\ Put this into a temporary file on the management server... cat <<'EOCAT' >/tmp/tsmchk.ksh export HN=$(hostname); for i in $(ps -ef | grep [[p]]mon | awk -F_ '{print $NF}'); do export SID=${i}; perl -00 -n -e 'print "$ENV{'HN'}|$ENV{'SID'}|$1\ " if m/TSM_$ENV{'SID'}_TDPO.*?erveraddress\\s+([[\\w-]]+)?/s' /etc/tsm/dsm.sys; done EOCAT chmod +x /tmp/tsmchk.ksh then send it to, and run it on, all the satellite servers... /home/ibmtools/scripts/oracle/doscp -f /tmp/tsmchk.ksh /home/ibmtools/scripts/oracle/dosh2 -c "ksh /tmp/tsmchk.ksh" This spits out a list of host | Instance | TSM server\\ If this list is pasted into a spreadsheet, a formula in Column 4 will show any bad ones (still backed up to the local data centre). ====== IF( MID(A1,3,2) = (MID(C1,4,2) ) ,"<===","") Obviously this is customer specific! Character positions 3-4 of the hostname show where the server is hosted, positions 4-5 of the TSM server name show its location. ==== Show details about all management classes on the TSM server ==== dsmc query mgmtclass -detail ==== Query running sessions on a TSM node ==== dsmc q sess -se=SO_U_SOLAX005_ORX_D_SOL and on Windows... dsmc q sess -optfile='C:\\home\\ibmtools\\etc\\oracle\\SO_I_WDCRHBP05_ORX_D_SOL.opt' ==== Find the default management class on the domain ==== dsmc q mgmt | grep 'Default Mgmt Class Name' | awk -F' ' '{print $NF}' ==== Force a management class ==== The TSM domain has a default management class. This can be overridden in both the dsm.opt file (include options) and on the command line.\\ Archiving files: dsmc archive -archmc=M14 -se=SO_U_`hostname`_ORX_D_BAX "/oracle/export/NSWORKP1/export_NSWORKP1_D_FULL_201301030007_01.dmp" ==== Restart LANFree Storage agent ==== udevdba31:root[[/home/root]]# ps -ef|grep dsmsta root 13828246 1 0 Mar 26 - 512:39 /usr/tivoli/tsm/StorageAgent/bin/dsmsta root 29491300 62259284 0 11:58:29 pts/1 0:00 grep dsmsta udevdba31:root[[/home/root]]# kill 13828246 udevdba31:root[[/home/root]]# ps -ef|grep dsmsta root 41418862 62259284 0 11:58:48 pts/1 0:00 grep dsmsta udevdba31:root[[/home/root]]# cat /etc/inittab | grep stg autostgagnt:2:off:/usr/tivoli/tsm/StorageAgent/bin/rc.tsmstgagnt udevdba31:root[[/home/root]]# at now /usr/tivoli/tsm/StorageAgent/bin/rc.tsmstgagnt Job root.1366365724.a will be run at Fri Apr 19 12:02:04 DFT 2013. udevdba31:root[[/home/root]]# ps -ef|grep dsmsta root 20906202 62259284 0 12:02:28 pts/1 0:00 grep dsmsta root 64225302 3735904 0 12:02:16 - 0:01 dsmsta quiet udevdba31:root[[/home/root]]# ==== Update number of mount points for a node ==== tsm: TSMSRV1>upd node CLUBDEV maxnummp=4 ANR2063I Node CLUBDEV updated. ==== Start dsmc in administrator mode ==== dsmadmc -id=ibmdba -pa=$DSMADMPWD -se=baxmac Add -comma or -tab to command line in order to get the query results in a list format dsmadmc -id=ibmdba -pa=$DSMADMPWD -se=baxmac -comma ==== Check TSM sessions are running Lanfree ==== Message number 415 shows the relevant sessions dsmadmc -id=ibmdba -pa=$DSMADMPWD -se=baxnos -comma "select * from actlog where msgno=415 and message like '%SO_U_JDBEOP1_ORA_X_BAX%' order by DATE_TIME desc fetch first 1 rows only" DATE_TIME: 2015-12-30 00:49:42.000000 MSGNO: 415 SEVERITY: I MESSAGE: ANR0415I Session 1011613 proxied by STABENOUERP07MAC started for node SO_U_JDBEOP1_ORA_X_BAX. (SESSION: 1011613) Parsing the session id out of the message, we can query the summary to find more info. dsmadmc -id=ibmdba -pa=$DSMADMPWD -se=baxnos "select * from summary where number=1011613" START_TIME: 2015-12-30 00:49:42.000000 END_TIME: 2015-12-30 00:49:44.000000 ACTIVITY: BACKUP NUMBER: 1011613 ENTITY: SO_U_JDBEOP1_ORA_X_BAX COMMMETH: Tcp/Ip ADDRESS: 10.100.119.145:61762 SCHEDULE_NAME: EXAMINED: 0 AFFECTED: 1 FAILED: 0 BYTES: 262155 IDLE: 0 MEDIAW: 0 PROCESSES: 1 SUCCESSFUL: YES VOLUME_NAME: DRIVE_NAME: LIBRARY_NAME: LAST_USE: COMM_WAIT: 1 NUM_OFFSITE_VOLS: ==== Clean up export dump and log files if they are on tape ==== for i in `find /oracle/export -name "*dmp" -type f`; do sid=`echo $i | awk -F_ '{print $2}'` echo dsmc q ar -se=so_u_`hostname`_orx_d_bax $i dsmc q ar -se=so_u_`hostname`_orx_d_bax $i >/tmp/results.$$ if grep $i /tmp/results.$$; then echo rm -f $i rm -f $i fi done rm /tmp/results.$$