DOYENSYS Knowledge Portal




We Welcome you to visit our DOYENSYS KNOWLEDGE PORTAL : Doyensys Knowledge Portal




Saturday, June 23, 2018

12.1.0.2 Cluster alert log and trace/logfiles

Alert log and Trace Location centralized for Oracle 12.1.0.2 cluster environment :-

Troubleshooting a RAC issue is now made easier as we can easily locate all the required log/trace files under a centralized location. We need not jump around different directories to find the Clusterware related log files

From 12c,the Clusterware log/trace files are stored in the Automatic Diagnostic Repository (ADR) under the ADR_HOME location $ADR_BASE/diag/crs/`hostname`/crs.

#---// Finding ADR_HOME for Clusterware logs //---
myrac1 {/home/oracle}: adrci
ADRCI: Release 12.1.0.2.0 - Production on Wed Jan 13 04:01:00 2018
Copyright (c) 1982, 2014, Oracle and/or its affiliates.  All rights reserved.
ADR base = "/app/oracle"
adrci> show homes
ADR Homes:
diag/tnslsnr/rac01/mgmtlsnr
diag/rdbms/_mgmtdb/-MGMTDB
diag/crs/rac1/crs

New centralized Clusterware log location to identify the related Clusterware log/trace files.
Oracle stores all the Clusterware diagnostic log/trace files under trace directory of Clusterware ADR_HOME.
#---// Locating Clusterware centralized log repository //----
rac1 {/home/oracle}: ls -ld $ADR_BASE/diag/crs/`hostname`/crs/trace
drwxrwxr-x 2 oracle oinstall 876544 Jan 13 04:02 /app/oracle/diag/crs/rac1/crs/trace

#---// moving to Clusterware centralized log repository //----
rac01 {/home/oracle}: cd $ADR_BASE/diag/crs/`hostname`/crs/trace

#---// Clusterware alert log file //----

rac01 {/app/oracle/diag/crs/rac1/crs/trace}: ls -lrt alert.log
-rw-rw---- 1 oracle oinstall 1024107 Jan 13 04:04 alert.log

Cluster Ready Services (CRS) daemon/agent trace files
#---// crs daemon and agent trace files //----
rac1 {/app/oracle/diag/crs/rac1/crs/trace}: ls -lrt crs* | tail -6
-rw-rw---- 1 root   oinstall  1340371 Jan 13 04:03 crsd.trm
-rw-rw---- 1 root   oinstall  7238701 Jan 13 04:03 crsd.trc  
-rw-rw---- 1 oracle oinstall   739563 Jan 13 04:04 crsd_oraagent_oracle.trm
-rw-rw---- 1 oracle oinstall  4371314 Jan 13 04:04 crsd_oraagent_oracle.trc  
-rw-rw---- 1 root   oinstall   650733 Jan 13 04:04 crsd_orarootagent_root.trm
-rw-rw---- 1 root   oinstall  3565651 Jan 13 04:04 crsd_orarootagent_root.trc 

Cluster Synchronization Services (CSS) daemon trace files
#---// css daemon trace files //----
rac1 {/app/oracle/diag/crs/rac1/crs/trace}: ls -lrt ocssd* | tail -2
-rw-rw---- 1 oracle oinstall   443750 Jan 13 04:04 ocssd.trm
-rw-rw---- 1 oracle oinstall  2868877 Jan 13 04:04 ocssd.trc

High Availability Service (HAS) daemon/agent trace files
#---// has daemon and agent trace files //----
rac1 {/app/oracle/diag/crs/rac1/crs/trace}: ls -lrt ohasd* | tail -6
-rw-rw---- 1 root   oinstall   783540 Jan 13 04:04 ohasd_orarootagent_root.trm
-rw-rw---- 1 root   oinstall  4805435 Jan 13 04:04 ohasd_orarootagent_root.trc
-rw-rw---- 1 root   oinstall   211308 Jan 13 04:04 ohasd.trm
-rw-rw---- 1 root   oinstall  1225033 Jan 13 04:04 ohasd.trc
-rw-rw---- 1 oracle oinstall  1382492 Jan 13 04:04 ohasd_oraagent_oracle.trm
-rw-rw---- 1 oracle oinstall  6877368 Jan 13 04:04 ohasd_oraagent_oracle.trc

Event Management (EVM) daemon trace files.
#---// evm daemon trace files //----
rac1 {/app/oracle/diag/crs/rac1/crs/trace}: ls -lrt evm* | tail -2
-rw-rw---- 1 oracle oinstall  1562665 Jan 13 04:04 evmd.trm
-rw-rw---- 1 oracle oinstall 10052660 Jan 13 04:04 evmd.trc

Grid Interprocess Communication (GIPC) daemon trace files
#---//
#---// gipc daemon trace files //----
#---// 
rac1 {/app/oracle/diag/crs/rac1/crs/trace}: ls -lrt gip* | tail -2
-rw-rw---- 1 oracle oinstall   763055 Jan 13 04:04 gipcd.trm
-rw-rw---- 1 oracle oinstall  5507597 Jan 13 04:04 gipcd.trc

Cluster System Monitor daemon trace files
#---//
#---// sysmon daemon trace files //----
#---// 
rac1 {/app/oracle/diag/crs/rac1/crs/trace}: ls -lrt osysmon* | tail -2
-rw-rw---- 1 root oinstall  557926 Jan 13 03:46 osysmond.trm
-rw-rw---- 1 root oinstall 2867760 Jan 13 03:46 osysmond.trc

Cluster Logger Service daemon trace files
#---//
#---// logger daemon trace files //----
#---// 
rac1 {/app/oracle/diag/crs/rac1/crs/trace}: ls -lrt ologgerd* | tail -2
-rw-rw---- 1 root oinstall   410752 Jan 13 03:48 ologgerd.trm
-rw-rw---- 1 root oinstall  2742627 Jan 13 03:48 ologgerd.trc

Apart from the Clusterware service (daemon) trace files, the diagnostic location also contains the client utility trace files as shown below
OCRCHECK utility trace files
#---//
#---// ocrcheck client trace files //----
#---// 
rac1 {/app/oracle/diag/crs/rac1/crs/trace}: ls -lrt ocrcheck* | tail -2
-rw-rw---- 1 oracle oinstall 137 Jan 13 01:30 ocrcheck_31338.trm
-rw-rw---- 1 oracle oinstall 310 Jan 13 01:30 ocrcheck_31338.trc

OCRCONFIG utility trace files.
#---//
#---// ocrconfig client trace files //----
#---// 
rac1 {/app/oracle/diag/crs/rac1/crs/trace}: ls -lrt ocrconfig* | tail -2
-rw-rw---- 1 oracle oinstall 137 Jan 13 01:30 ocrconfig_30866.trm
-rw-rw---- 1 oracle oinstall 312 Jan 13 01:30 ocrconfig_30866.trc

CRSCTL utility trace files.
#---//
#---// crsctl client trace files //----
#---// 
rac1 {/app/oracle/diag/crs/rac1/crs/trace}: ls -lrt crsctl* | tail -2
-rw-rw---- 1 oracle oinstall  175 Jan 13 01:31 crsctl_1103.trm
-rw-rw---- 1 oracle oinstall  729 Jan 13 01:31 crsctl_1103.trc

Cluster Monitor utility (oclumon) trace files
#---//
#---// oclumon client trace files //----
#---// 
rac1 {/app/oracle/diag/crs/rac1/crs/trace}: ls -lrt oclumon* | tail -2
-rw-rw---- 1 oracle oinstall 166 Dec 17 03:54 oclumon_2175.trm
-rw-rw---- 1 oracle oinstall 387 Dec 17 03:54 oclumon_2175.trc

Grid Plug and Play utility (GPNP) trace files
#---//
#---// gpnp client trace files //----
#---// 
rac1 {/app/oracle/diag/crs/rac1/crs/trace}: ls -lrt gpn* | tail -2
-rw-rw---- 1 oracle oinstall  205 Jan 13 01:31 gpnptool_664.trm
-rw-rw---- 1 oracle oinstall  608 Jan 13 01:31 gpnptool_664.trc

Therefore,Clusterware trace/log files are placed under a single centralized log repository. From 12c Release 1,we can troubleshooting Clusterware related issues easily under one location.

No comments: