Thursday, 2 January 2020

HA Database Upgrade from 11.2.0.3/4 to 12.1.0.2


Preinstallation


  • Create CRQ for the upgrade
  • Copy the required files
  • Copy PSU  and OPatch to HA Node
  • Unzip the database files
  • Create RDBMS response file
  • Deploy software
  • Copy init.ora and tnsnames.ora to new RDBMS Home
  • Upgrade OPatch 
  • Create ocm response file
  • Update  config.c with ASM group
  • Apply latest PSU
  • Update Goldengate Library
  • Verify Installation
  • Verify a good backup
  • Pre-Upgrade check

Run root.sh

Upgrade the database

  • Run Pre-Upgrade SQL
  • Disable Golden Gate  System Trigger
  • Shutdown Database
  • Update /etc/oratab
  • Startup Upgrade
  • Upgrade the Data Dictionary
  • Check the upgrade for errors
  • Run Post-Upgrade SQL
  • Enable GoldenGate  System Trigger
  • Apply PSU  to Database and recompile
  • Verify PSU
  • Verify correct dump destination
  • Shutdown Database
  • Update OCR and start 
  • Compile system triggers
  • Verify Oratab file
  • Update DST
  • Close CRQ and Release  Database

*************************************************************************************************************

Preinstallation

Create CRQ for the Upgrade

Copy the required files 

These instructions assume  the following files are copied to /oraworkspace/software location

RDBMS Software Files
p17694377_121020_Linux_x86-64_1of8.zip(rdbms #1)
p17694377_121020_Linux_x86-64_2of8.zip(rdbms #2)
Latest PSU

p20299023_121020_Linux-x86-64.zip (PSU 3-copy to HA Node)

Latest OPatch

p6880880_121010_Lunix-x86-64.zip (OPatch -copy to HA Node)




Unzip the database files
As Oracle:

cd /oraworkspace/software
unzip p17694377_121020_Linux_x86-64_1of8.zip
unzip p17694377_121020_Linux_x86-64_2of8.zip

Create RDBMS response files

As Oracle:

cd /oraworkspace/software
grep -i ^[A-Z] database/response/db_install.rsp | sort > dbinstall.rsp

Modify the following variables as per environment you are patching

DECLINE_SECURITY_UPDATES=true
INVENTORY_LOCATION=/u01/app/oraInventory
ORACLE_BASE=/u01/app/oracle
ORACLE_HOME=/u01/app/oracle/product/12.1.0.2/PSU3
oracle.install.db.BACKUPDBA_GROUP=oinstall
oracle.install.db.DBA_GROUP=dba
oracle.install.db.DGDBA_GROUP=oinstall
oracle.install.db.EEOptionsSelection=true
oracle.install.db.InstallEdition=EE
oracle.install.db.KMDBA_GROUP=oinstall
oracle.install.db.OPER_GROUP=oper
oracle.install.option=INSTALL_DB_SWONLY
UNIX_GROUP_NAME=oinstall

Deploy Software

As Oracle

cd /oraworkspace/software/database

./runInstaller -silent -responseFile /oraworkspace/software/dbInstall.rsp

Copy init.ora and tnsnames.ora to new RDBMS Home


As Oracle 


. oraenv
cp - p $ORACLE_HOME/network/admin/tnsnames.ora /u01/app/oracle/product/12.1.0.2/PSU3/network/admin

For Each Database:

.oraenv

cp - p $ORACLE_HOME/dbs/init${ORACLE_SID}.ora ${ORACLE_HOME}/dbs/orapw${ORACLE_SID} \  /u01/app/oracle/product/12.1.0.2/PSU3/dbs
done

Upgrade OPatch 


As Oracle

cd /u01/app/oracle/product/12.1.0.2/PSU3
unzip -o /oraworkspace/software/p6880880_121010_Linux-x86-65.zip

Create OCM response file:

Check for /var/tmp/ocm.rsp if doesn't exist

As Oracle

The following command has 2 Prompts


1. Email address/User Name: Leave Blank
2. Do you wish to remain uninformed ofsecurity issues(Yes,No)N:Answer 'Y"

export ORACLE_HOME=/u01/app/oracle/product/12.1.0.2/PSU3
${ORACLE_HOME}/OPatch/ocm/bin/emocmrsp -no_banner -output /var/tmp/ocm.rsp


Example Output:

${ORACLE_HOME}/OPatch/ocm/bin/emocmrsp -no_banner -output /var/tmp/ocm.rsp
Provide  your email address to be informed of security issues, install and initiate oracle configuration  Manager. Easier for you if you use your My Oracle Support Email address/User Name.

Visit http://www.oracle.com/support/policies.html for details
Email address/Username:

You have not provided an email address for notification of security issues.
Do you Wish to remain  uninformed of security issues([Y]es,[N]o [N]:Y

The ocm configuration  response file  (/var/tmp/ocm.rsp)was successfully created.

Update config.c with ASM group 

Correct the value of ASM group in the  $ORACLE_HOME/rdbms/lib/config.c file 

This will result in ASM  group  set after the PSU  relink in the following steps

As Oracle


export ORACLE_HOME=/u01/app/oracle/product/12.1.0.2/PSU3
sed -t 's/SS_ASM_GRP ""/SS_ASM_GRP "asmadmin"/' ${ORACLE_HOME}/rdbms/lib/config.c

Apply latest PSU


As Oracle


. If the grid patch will be used 

cd /oraworkspace/software
export ORACLE_HOME=/u01/app/oracle/product/12.1.0.2/PSU3
${ORACLE_HOME}/OPatch/opatch apply /orworkspace/software/gpsu/20485724/20299023 -oh ${ORACLE_HOME} -ocmrf /var/tmp/ocm.rsp -silent

. Ifthe RDBMS patch will be used


cd /oraworkspace/software
mkdir psu
cd psu
unzip ../p20299023_121020_Linux-x86-64.zip
export ORACLE_HOME=/u01/app/oracle/product/12.1.0.2/PSU3

${ORACLE_HOME}/OPatch/opatch apply /orworkspace/software/psu/20299023 -oh ${ORACLE_HOME} -ocmrf /var/tmp/ocm.rsp -silent

Verify the output mentions completed.

Example:


All nodes have been patched. You may start oracle instances on the local system and nodes

Composite patch 20299023 successfully applied .
OPatch session completed with warnings.
log file location: /u01/app/oracle/product/12.1.0.2/PSU3/cfgtoollogs/opatch/opaatch2019-12-30_18-56-12PM_1.log
OPatch completed with warnings.

Update Goldengate Library


As Oracle


. oraenv
ln -s ${ORACLE_HOME}/lib/libnnz12.so ${ORACLE_HOME}/lib/libnnz11.so

Verify Installation


As Oracle:


Verify software installed 


du -sk /u01/app/oracle/product/12.1.0.2/PSU3



Should looks something like the  following . Verify the directory  size is similar.

6260832 /u01/app/oracle/product/12.1.0.2/PSU3

Check the inventory to insure all appears to be in order

tail /u01/app/oraInventory/ContentsXML/inventory.xml


Verify PSU applied 


/u01/app/oracle/product/12.1.0.2/PSU3/OPatch/opatch_lsinventory

Look for a line similar to below

Patch Description: "Database Patch set Update:12.1.0.2:3 (20299023)"

Verify a good backup


Before any change is made to the database , Insure a good backup is in place.

. Check the crontab for the regularly scheduled backup
. Check the logfile to insure the last backup was successful.


. If backup is not scheduled or the backups are not successful

Run export backup and create guarantee restore point.


Pre-Upgrade Check


Run the pre-upgrade check on the database to upgrade . Will need to be  run individually for each database to upgrade.

As Oracle for each database

Set the oracle 11g environment

# Run for each instance to be upgraded.

. oraenv

Run the Oracle  12c Pre-Upgrade generation script(From the oracle 12c home)

sqlplus /nolog <<EndSQL
connect / as sysdba
@/u01/app/oracle/product/12.1.0.2/PSU3/rdbms/admin/preupgrd.sql
exit
EndSQL

Run the generated pre-upgrade 

sqlplus /nolog <<EndSQL
connect / as sysdba
purge dba_recyclebin;
column NAME new_value DBName
select name from v\$database;
@/u01/app/oracle/cfgtoollogs/<DBName>/preupgrade/preupgrade_fixups.sql
EndSQL

Review the output from the above command for each database . Will need  to run the required items either before or after the upgrade

./u01/app/oracle/cfgtoollogs/<DBName>/preupgrade/preupgrade.log
./u01/app/oracle/cfgtoollogs/<DBName>/preupgrade/preupgrade_fixups.sql
./u01/app/oracle/cfgtoollogs/<DBName>/preupgrade/postupgrade_fixups.sql

Run root.sh

AS root
/u01/app/oracle/product/12.1.0.2/PSU3/root.sh


Example Output:

Check /u01/app/oracle/product/12.1.0.2/PSU3/install/root_10.121.23.12_2019_12_31_20-31-12.log for the output of root script

Upgrade the database




Run Pre-Upgrade SQL


Run the Pre-Upgrade script and the manual items from preupgrd.sql executed above

As Oracle


#Run each instance to be upgraded . oraenv

sqlplus /nolog <<EndSQL
connect / as sysdba
purge dba_recyclebin;
column Name new_value DBName
select name from v\$database;
@/u01/app/oracle/cfgtoollogs/&DBName./preupgrade/preupgrade_fixups.sql
EndSQL

Note that the Preupgrd_fixups.sql script doesn't execute all required items. The "MANUAL ACTION REQUIRED" manually.

Disable Golden Gate System Trigger


sqlplus /nolog <<EndSQL
connect / as sysdba
alter trigger sys.GGS_DDL_TRIGGER_BEFORE disable;
EndSQL

The above will fail if golden gate is not installed. Any error can be ignored.


Shutdown Database


As Oracle


. oraenv
srvctl stop database -d <DBNAME>

Update /etc/oratab


As Oracle, Update /etc/oratab file to the new ORACLE_HOME 

Startup Upgrade


Startup the database using new ORACLE_HOME


As Oracle


. oraenv

sqlplus /nolog <<EndSQL
connect / as sydba
Startup Upgrade
EndSQL

Upgrade the Data Dictionary


As Oracle

. oraenv

cd $ORACLE_HOME/rdbms/admin
$ORACLE_HOME/perl/bin/perl/ catctl.pl -n 8 catupgrd.sql


Example Output


Argument list for  [catctl.pl]
SQL Process Count     n=8
SQL PDB Prcess Count  N=0
Input Directory       d=0
Phase Logging Table   t=0
Log Dir               l=0
Script                s=0    
$ORACLE_HOME/perl/bin/perl/ catctl.pl -n 8 -l ${ORACLE_HOME}/rdbms/log catupgrd.sql
...
catctl.pl version 12.1.0.2
Oracle Base       =/u01/app/oracle
Analyzing file catupgrd.sql
Log files in  /u01/app/oracle/product/12.1.0.2/PSU3/rdbms/admin
catcon: ALL catcon-related output will be written to catupgrd_catcon_30168.lst
catcon: See catupgrd*.log files for output generated by script
catcon: See capupgrd_*.lst files for spool files, if any 

--------------------------------------------------------------
Phases [0-73]
Serial  Phase #: 0 Files: 1 Time: 103s
Serial  Phase #: 1 Files: 5 Time: 44s
.....................................
.....................................
.....................................
Serial  Phase #: 72 Files: 1 Time: 0s
Serial  Phase #: 73 Files: 5 Time: 23

Grand Total Time: 952s
LOG FILES: (Catupgrd*.log)
Upgrade Summary  Report Located in :
/u01/app/oracle/product/12.1.0.2/PSU3/cfgtoollogs/DBNAME/upgrade/upg_summary.log
Grand Total Upgrade Time: [0d:0h:15m:52s]

Check Upgrade for errors


As Oracle 

cat /u01/app/oracle/product/12.1.0.2/PSU3/cfgtoollogs/SPS/upgrade/upg_summary.log

Example Output:

Function Created.

Elapsed:00:00:00:20
Oracle Database 12.1 Post-Upgrade Status Tool       12-31-2019 12:53:32

Component Name                            Current Status      Version Number    Elapsed  Time HH:MM:SS  
Oracle Server                             UPGRADED      12.1.0.2.0    00:14:50
JServer JAVA Virtual Machine              VALID         12.1.0.2.0    00:04:22
Oracle Workspace Manager                  VALID         12.1.0.2.0    00:01:55
Oracle XDK                                VALID         12.1.0.2.0    00:01:09
Oracle XML Database                       VALID         12.1.0.2.0    00:01:43
Oracle Database Java Packages             VALID         12.1.0.2.0    00:00:43
Final Actions                                                         00:01:28
Post Upgrade                                                          00:10:23
Total Upgrade Time :00:37:49
PL/SQL procedure successfully completed.
Elapsed :00:00:00:62
Function Dropped.
Elapsed :00:00:00:14
Grand Total Upgrade Time:   [0d:0h:38m:37s]            

Run Post Upgrade SQL


Run the post-upgrade script and manual items from the preupgrd.sql executed above 

As Oracle


. oraenv
sqlplus /nolog <<EndSQL
connect / as sysdba
purge dba_recyclebin;
column Name new_value DBName
select name from v\$database;
@/u01/app/oracle/cfgtoollogs/&DBName./preupgrade/postupgrade_fixups.sql
EndSQL

Enable Goldengate System Trigger



sqlplus /nolog <<EndSQL
connect / as sysdba
alter trigger sys.GGS_DDL_TRIGGER_BEFORE enable;
EndSQL

The above will fail if golden gate is not installed. Any error can be ignored.

Apply PSU to Database and Recompile


Check for invalid objects and run the final upgrade verification script

As oracle

. oraenv
sqlplus /nolog <<EndSQL
connect / as sysdba
startup
EndSQL
${ORACLE_HOME}/OPatch/datapatch -apply 20299023

sqlplus /nolog <<EndSQL
connect / as sysdba
@?/rdbms/admin/utlrp.sql
@?/rdbms/admin/utluiobj.sql
@?/rdbms/admin/utlul21s.sql
EndSQL

Verify PSU


As Oracle 


. oraenv
sqlplus /nolog <<EndSQL

connect / as sysdba

set linesize 100
column banner format a80
column version format a10
column action format a10
column status format a10
column description format a50
select * from v\$version;
select patch_id,version,action,status,description from dba_registry_sqlpatch order by action_time;
EndSQL


Verify Correct Dump Destinations


Be Sure Diagnostic_dest is properly set and *dump_dest  default

As Oracle


. oraenv
sqlplus /nolog <<EndSQL
connect / as sysdba
show parameter diagnostic_dest

alter  system set diagnostic_dest='/oratrace' scope=spfile sid='*';

show parameter background_dump_dest


alter  system reset background_dump_dest scope=spfile sid='*';


show parameter core_dump_dest



alter  system reset core_dump_dest scope=spfile sid='*';

show parameter user_dump_dest




alter  system reset user_dump_dest scope=spfile sid='*';

EndSQL

Anyone of the above may result in a error that can be ignored.

.ORA-32010: Cannot find entry to delete in SPFILE



Set correct audit_file_dest and insure directories exist




sqlplus /nolog <<EndSQL
connect / as sysdba
column C1 new_name C!
select  'set audit_file_dest=''/oradbaudit/admin//'||name||'/adump''scope=spfilesid=''*''' C1 from v\$database;

alter system &C1.
/
set pagesize 0
set linesize 120
select 'ssh -q ' ||host_name||" mkdir -p /oradbaudit/admin/' ||name||'/adump from v\$database,gv\instance;
EndSQL


Run the above 'ssh -q..." statements from the command line

Shutdown Database

As Oracle
. oraenv


sqlplus /nolog <<EndSQL
connect / as sysdba
shut immediate;
EndSQL


Update OCR and start


As Oracle


. oraenv

srvctl upgrade database -d <DBNAME> -o /u01/app/oracle/product/12.1.0.2/PSU3
srvctl start database d <DBName>

Compile System Triggers


As Oracle

. oraenv


sqlplus /nolog <<EndSQL
connect / as sysdba
select 'alter trigger '||owner||','||object_name||' compile;' from dba_objects where status='INVALID' and object_type='TRIGGER';
EndSQL

Run the above "alter trigger...."statements from the command line

Verify Oratab file


As Oracle


verify /etc/oratab properly updated on all nodes

Update DST


Update DST to version 18 Follow below link to instructions.
Oracle DST 18 Update

No comments:

Post a Comment