Skip to main content

11g Quick Steps to Package and Send Critical Error Diagnostic Information to Support


11g Quick Steps to Package and Send Critical Error Diagnostic Information to Support

 Modified 19-SEP-2011     Type HOWTO     Status PUBLISHED 

In this Document
  Goal
  Solution
  References




Applies to:

Oracle Server - Enterprise Edition - Version: 11.1.0.6 to 11.2.0.2 - Release: 11.1 to 11.2
Information in this document applies to any platform.

Goal

Starting with Oracle 11g, Alert.log and trace file are generated in a new format and are located in Automatic Diagnostic Repository ( ADR ).
Refer to the Document 422893.1 to know ADR in detail.
This article explains how to find and send the alert.log and relevant trace files to support when a critical error occur in database, using ADRCI tool.

Solution

Oracle 11g provides tools and methods to automatically identify and gather trace files related to a problem (a critical error in database) or an incident (An occurrence of a problem). This facility is known as Incident Packaging Services (IPS). Support Workbench (GUI) and ADRCI (Command Line) are the two interfaces to Incident Packaging Services.
Every occurrence of a critical error (problem) in the database creates an incident.
Steps in identifying and sending files to support.
1. You are notified of a critical error reported in the database
SQL> select * from atab;
select * from atab
*
ERROR at line 1:
ORA-01578: ORACLE data block corrupted (file # 6, block # 11)
ORA-01110: data file 6: '/opt/oracle/oradata/db11g/tt.dbf' 
2. Check for the error reported in the Alert.log and in the ADR .
To check for the error reported in ADR :
Invoke adrci :
with 11g Environment set , from the OS prompt
%]adrci
Set the home path :
adrci>show home
--show home will list all the ADR HOMES. set the database ADR home.
adrci>set homepath database home 
Check for the problem reported
adrci> show problem

ADR Home = /opt/oracle/diag/rdbms/db11g/db11g:
*************************************************************************
PROBLEM_ID PROBLEM_KEY LAST_INCIDENT LASTINC_TIME
-------------------- ----------------------------------------------------------- -------------------- ----------------------------------------
1 ORA 1578 18104 2009-06-01 22:06:19.501207 +10:00
1 rows fetched 

You can have multiple problems reported . Create a package for the one which needs to be investigated .

3. Check for the incidents reported for the problem which needs to be packaged
Each occurance of the problem is called 'incident'. Alert.log will show all the incidents reported .  Each incident will be associated with a unique incident id .
From ADR, execute 'show incident' command  for the required problem key. You can get the problem key from 'show problem' command output :
adrci> show incident -p "problem_key='ORA 1578'"

ADR Home = /opt/oracle/diag/rdbms/db11g/db11g:
*************************************************************************
INCIDENT_ID PROBLEM_KEY CREATE_TIME
-------------------- ----------------------------------------------------------- ----------------------------------------
18147 ORA 1578 2009-06-01 22:02:08.805002 +10:00 
You can see multiple incidents reported for the same problem.
Note the incident number for which you need to create a package.

4. Package the trace files invoking IPS .
Generate the package using 'IPS pack' command in the specified path . Example below shows the package for the incident created in /tmp.
adrci> ips pack incident 18147 in /tmp
Generated package 9 in file /tmp/ORA1578_20090602113045_COM_1.zip, mode complete 

Examples of IPS PACK :
ips pack problem 100 in /tmp
--Generates the package for the problem id 100 in /tmp


ips pack incident 6439 in /tmp
--Generates the package for the incident id 6439 in /tmp


ips pack problemkey "ORA 1578"
--Generates the package for the problem with the problem_key 'ORA 1578'


ips pack seconds 8
--Generates the package with the incidents occured in last 8 seconds.

ips pack time '2007-05-01 10:00:00.00' to '2007-05-01 23:00:00.00'
--Generates the package with the incidents occured between the times '2007-05-01 10:00:00.00' and '2007-05-01 23:00:00.00'

Comments

Popular posts from this blog

ORA-65139: Mismatch between XML metadata file and data file

Error: CDB$ROOT@EBTUPRDC> CREATE PLUGGABLE DATABASE EBTUSPRDPDB1 using '/home/oracle/EBTUSPRDC.xml' nocopy tempfile reuse; CREATE PLUGGABLE DATABASE EBTUSPRDPDB1 using '/home/oracle/EBTUSPRDC.xml' nocopy tempfile reuse * ERROR at line 1: ORA-65139: Mismatch between XML metadata file and data file /u05/odb/ORADATA_1/EBTUSPRDC/datafile/system.1531.807800993 for value of fcpsb (2230502002 in the plug XML file, 2230505492 in the data file) Fix: Don't open Non-CDB database until we complete pdb creation other wise we will get above error. Shutdown immediate; startup mount; alter database open read only; create xml file. shutdown immediate; Connect to CDB and Create Pluggable database .

Useful OEM Queries to get Target details from OEM Repository

List Targets with TNS Listener ports configured : SELECT mgmt$target.host_name , mgmt$target.target_name , mgmt$target.target_type , mgmt$target_properties.property_name , mgmt$target_properties.property_value FROM mgmt$target , mgmt$target_properties WHERE ( mgmt$target.target_name = mgmt$target_properties.target_name ) AND ( mgmt$target.target_type = mgmt$target_properties.target_type ) and ( mgmt$target.target_type = 'oracle_listener' ) and ( mgmt$target_properties.property_name = 'Port' ); Devora02       LISTENER_ora02                       oracle_listener Port 1529 Devora01       LISTENER_ora01                       oracle_listener Port 1529 Devora04       LISTENE...

TFA-00002 : Oracle Trace File Analyzer (TFA) is not running

TFA Failed to start listening for commands on one of the Rac Node : As a root user /etc/init.d/init.tfa  start Starting TFA.. start: Job is already running: oracle-tfa Waiting up to 100 seconds for TFA to be started.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Successfully started TFA Process.. . . . . . TFA-00002 : Oracle Trace File Analyzer (TFA) is not running TFA Failed to start listening for commands Solution : run synctfanodes.sh  if you have root password . if no root password copy below files to TFA Failed node and try to restart TFA .. Please check whether these 4 files exist on all nodes are similar. TFA_HOME/server.jks TFA_HOME/client.jks TFA_HOME/internal/ssl.properties   TFA_HOME/internal/ portmapping.txt If not similar just copy these files to TFA Failing node and try TFA Restart ..It Worked for me .   [root@host01]# ./tfactl  -v "debug" start Starting TFA...