Difference between revisions of "Patching Oracle Database Appliance (ODA)"
| Line 1: | Line 1: | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
===Patch preparation=== | ===Patch preparation=== | ||
Show version details of all ODA components | Show version details of all ODA components | ||
Revision as of 15:40, 14 December 2016
Contents
- 1 Patch preparation
- 2 Patch the ODA infrastructure
- 2.1 Remove any non-standard RPMs
- 2.2 Review size of shared repository
- 2.3 Ready for patching...
- 2.4 Stop Snap management
- 2.5 Shutdown any virtual machines
- 2.6 Shutdown Grid agent / Listeners
- 2.7 Shutdown and backup ODA_BASE
- 2.8 Optonally reconfigure ODA_BASE
- 2.9 Restart ODA_BASE
- 2.10 Apply the ODA infrastructure patch
- 2.11 Run post infra patch checks
- 3 Patch the Grid infrastructure
- 4 Patch the RDBMS
- 5 Post ODA patch activities
- 5.1 Restart any physical standby databases
- 5.2 Restart RAC cluster
- 5.3 Verify all ODA components are now up-to-date
- 5.4 Restart any virtual machines
- 5.5 Remount ZFS volumes on virtual machines
- 5.6 Restart OMS
- 5.7 Check ODA Grid agent
- 5.8 Reinstall any previously uninstalled RPMs
- 5.9 Uncomment any specific Oracle commands
- 5.10 Uncomment any jobs that clean up archivelogs
- 5.11 Take databases out of blackout on OEM
- 5.12 Check public keys
- 6 Useful queries
Patch preparation
Show version details of all ODA components
/opt/oracle/oak/bin/oakcli show version -detail
Patching ODA means patching the O/S, the Grid Infrastructure (GI) and the RDBMS.
Useful ODA documents:
- Database Appliance Administration and Reference Guide
- Master ODA document
- Oracle Database Appliance - 12.1.2 and 2.X Supported ODA Versions & Known Issues (Doc ID 888888.1)
- 12.1.0.2 Proactive Database Bundle Patches
Download latest ODA patch bundle
Latest ODA patches are found in Oracle Database Appliance - 12.1.2 and 2.X Supported ODA Versions & Known Issues (Doc ID 888888.1)
Download and copy zip files
Download the files on the relevant patch page for your hardware, copy them to wherever you store patches on the server and Read the README.
Copy zip files to /tmp
Copy the zip files from your patch storage area to /tmp on both ODA_BASE nodes.
Check space
Make sure there is enough space in /root, /tmp, $ORACLE_HOME, and /opt for the unzippped / installed files.
Verify ODA version
/opt/oracle/oak/bin/oakcli show version
Unpack zip files
Do not use unzip!
cd /tmp oakcli unpack –package p25064836_121290_Linux-x86-64_1of2.zip oakcli unpack –package p25064836_121290_Linux-x86-64_2of2.zip
Delete zips from /tmp
If unpack -package was successful, files can be removed from /tmp.
rm -rf /tmp/p25064836_121290_Linux-x86-64*
Check what components will be patched
/opt/oracle/oak/bin/oakcli update -patch 12.1.2.9.0 -verify
Backup sysctl.conf
export DATESTAMP=`date +%Y%m%d`
cp /etc/sysctl.conf /etc/sysctl.conf_${DATESTAMP}
Patch the ODA infrastructure
Remove any non-standard RPMs
oakcli validate -c ol6upgrade -prechecks
If this check reveals any RPMs, they must be removed prior to patching. Note: Perform check on both nodes, there may be differences!
rpm -e --nodeps libtermcap-devel.x86_64 rpm -e --nodeps yum-downloadonly.noarch rpm -e --nodeps gpg-pubkey rpm -e --nodeps check_mk-agent.noarch rpm -e --nodeps readline-devel.x86_64
oakcli show repo
and add space if necessary
oakcli configure repo srepo -incsize 100G
Ready for patching...
Perform any specifics such as
- Set database in blackout in OEM
- Any other monitoring software?
- Ensure archivelogs are not deleted. They will be needed for application on physical standby later.
- anything else that could be a problem. These aliases won't work any more as we just uninstalled the readline package so comment them for now.
cd vi .bash_profile #alias sqlplus='rlwrap -D2 -ic sqlplus' #alias dgmgrl='rlwrap -D2 -ic dgmgrl' #alias rman='rlwrap -D2 -ic rman' #alias asmcmd='rlwrap -D2 -ic asmcmd' #alias lsnrctl='rlwrap -D2 -ic lsnrctl' #alias adrci='rlwrap -D2 -ic adrci' #alias impdp='rlwrap -D2 -ic impdp' #alias expdp='rlwrap -D2 -ic expdp'
Stop Snap management
/opt/oracle/smu/bin/smu stop
Shutdown any virtual machines
oakcli stop vm odavmdp1 -force oakcli stop vm odavmdp2 -force oakcli stop vm odavmdb3 -force oakcli stop vm odavmoms1 -force
Shutdown Grid agent / Listeners
export ORACLE_SID=AGENT ORAENV_ASK=NO . oraenv emctl stop agent export TNS_ADMIN=$ORACLE_HOME/network/admin/ lsnrctl stop LISTENER_DATAGUARD
Shutdown and backup ODA_BASE
Note: Do this on both nodes!
oakcli stop oda_base cd /OVS tar -cvzf oakDom1.odavm01-rac.tar.gz /OVS/Repositories/odabaseRepo/VirtualMachines/oakDom1
Optonally reconfigure ODA_BASE
oakcli configure oda_base
Restart ODA_BASE
After tarball has been successfully created, ODA_BASE can be restarted.
oakcli start oda_base
Apply the ODA infrastructure patch
Note: Do NOT run this in parallel on both nodes. The idea is that one of the nodes remains up and usable while patching the other!
/opt/oracle/oak/bin/oakcli update -patch 12.1.2.9.0 --infra --local
Run post infra patch checks
/opt/oracle/oak/bin/oakcli validate -c ol6upgrade –postchecks sudo -u root -i su - grid $GRID_HOME/bin/crsctl check crs /opt/oracle/oak/bin/oakcli show version -detail
Patch the Grid infrastructure
oakcli stop repo srepo oakcli show repo
Apply the GI patch
Carefully read the known issues before applying the GI Patch.
/opt/oracle/oak/bin/oakcli update -patch 12.1.2.9.0 --gi
Check cluster health
${ORACLE_HOME}/grid/bin/crsctl check crs
Change some file permissions
chmod 755 ${ORACLE_HOME}/grid/bin/osdbagrp ${ORACLE_HOME}/grid/lib/acfsreplcrs.pl
Restore sysctl.conf
Check for higher values of variables.
${ORACLE_HOME}/grid/bin/crsctl check crs
Check cluster health
${ORACLE_HOME}/grid/bin/crsctl check crs
Patch the RDBMS
Apply the database patch
Once the infrastructure and GI patches have been completed successfully, the RDBMS patches can be applied.
It can be applied locally on each node or as a rolling upgrade and both nodes will be patched automatically.
/opt/oracle/oak/bin/oakcli update -patch 12.1.2.9.0 --database <-local>
Verify database versions
oakcli show dbhomes -detail oakcli show databases -detail
Post ODA patch activities
Restart any physical standby databases
Restart RAC cluster
svrctl start database
Verify all ODA components are now up-to-date
/opt/oracle/oak/bin/oakcli show version -detail
Restart any virtual machines
oakcli start vm odavmdp1 oakcli start vm odavmdp2 oakcli start vm odavmdb3 oakcli start vm odavmoms1
Remount ZFS volumes on virtual machines
Log in to each of the VMs in turn
sudo mount /orasoft sudo mount /orabackups/RMAN sudo mount /orabackups/datapump_hist
Restart OMS
login to OMS server
cd /u01/OracleHomes/Middleware/oms/bin ./emctl status oms ./emctl start oms
Check ODA Grid agent
Kill any old processes and restart agent
export ORACLE_SID=agent ORAENV_ASK=NO . oraenv ps -ef | grep emagent kill -9 <pid where agent is 12.1.0.6> emctl start agent
Reinstall any previously uninstalled RPMs
Check for later versions?
- Nagios plugin
cd /orasoft/rpm yum install check-mk-agent-1.2.6p14-e23d6a3d4540a28a.noarch.rpm
- Oracle Snap Manager
cd /orasoft/oracle_snapmanager/oracle-smu-1.2.0-ga/Linux/noarch/ rpm -i oracle-smu-1.2.0-27.noarch.rpm /opt/oracle/smu/bin/smu start
- other
yum install libtermcap-devel.x86_64 yum install yum-downloadonly.noarch yum install gpg-pubkey yum install readline-devel.x86_64
Uncomment any specific Oracle commands
Like these aliases setup for readline wrapping
alias sqlplus='rlwrap -D2 -ic sqlplus' alias dgmgrl='rlwrap -D2 -ic dgmgrl' alias rman='rlwrap -D2 -ic rman' alias asmcmd='rlwrap -D2 -ic asmcmd' alias lsnrctl='rlwrap -D2 -ic lsnrctl' alias adrci='rlwrap -D2 -ic adrci' alias impdp='rlwrap -D2 -ic impdp' alias expdp='rlwrap -D2 -ic expdp'
Uncomment any jobs that clean up archivelogs
Take databases out of blackout on OEM
...and any other monitoring software
Check public keys
Patching may destroy public keys
ssh admin@oda02-rac -p 8002
If password is requested, keys are corrupt.
Useful queries
As seen on Pythian
prompt ------------------------------------------
prompt Patches applied to both the $OH and the DB
prompt ------------------------------------------
with a as (select dbms_qopatch.get_opatch_lsinventory patch_output from dual)
select x.patch_id
, x.patch_uid
, x.rollbackable
, s.status
, x.description
from a
, xmltable('InventoryInstance/patches/*'
passing a.patch_output
columns
patch_id number path 'patchID',
patch_uid number path 'uniquePatchID',
description varchar2(80) path 'patchDescription',
rollbackable varchar2(8) path 'rollbackable'
) x
, dba_registry_sqlpatch s
where x.patch_id = s.patch_id
and x.patch_uid = s.patch_uid
-- and s.bundle_series = 'PSU'
/
prompt --------------------------------------------------------
prompt Patches installed into the $OH but not applied to the DB
prompt --------------------------------------------------------
with a as (select dbms_qopatch.get_opatch_lsinventory patch_output from dual)
select x.patch_id
, x.patch_uid
, x.description
from a
, xmltable('InventoryInstance/patches/*'
passing a.patch_output
columns
patch_id number path 'patchID',
patch_uid number path 'uniquePatchID',
description varchar2(80) path 'patchDescription'
) x
minus
select s.patch_id
, s.patch_uid
, s.description
from dba_registry_sqlpatch s
/
prompt --------------------------------------------------------
prompt Patches applied to the DB but not installed into the $OH
prompt --------------------------------------------------------
with a as (select dbms_qopatch.get_opatch_lsinventory patch_output from dual)
select s.patch_id
, s.patch_uid
, s.description
from dba_registry_sqlpatch s
minus
select x.patch_id
, x.patch_uid
, x.description
from a
, xmltable('InventoryInstance/patches/*'
passing a.patch_output
columns
patch_id number path 'patchID',
patch_uid number path 'uniquePatchID',
description varchar2(80) path 'patchDescription'
) x
/