Oracle 10g/11g RAC Clusterware Installation Tips

Part II: Oracle RAC High Availability Series


Since I have been building clusters with Oracle 10g and 11g RAC environments for many
new clients, I have decided to present a series on Oracle RAC tips and tricks. The previous installment for Oracle RAC technology included design and implementation
guidelines for best practices. This week, we will cover the post installation tips
with the Oracle 10g/11g Cluster Verification Utility called cluvfy. This wonderful tool provides insight into whether or not your Oracle RAC environment has been installed correctly.

The Oracle Clusterware utility for Oracle RAC comes in two forms: the cluvfy utility which is available after installation of the clusterware software and the runcluvfy.sh shell script which can be used before installation. In the following example, we will use the post install verification feature available with cluvfy to verify our Oracle 10g RAC clusterware installation.


Verify Oracle RAC Clusterware Installation using the Clusterware Verification Utility (CLUVFY)


Cluvfy uses stages to check the status of the installation for before and after each
phase of the Oracle RAC installation process.

Below is the basic syntax for cluvfy:

oracle@racnode1 $ cluvfy


USAGE:
cluvfy [ -help ]
cluvfy stage { -list | -help }
cluvfy stage {-pre|-post} [-verbose]
cluvfy comp { -list | -help }
cluvfy comp [-verbose]

USAGE:
cluvfy stage -post crsinst -n [-verbose]

is the comma separated list of non-domain qualified nodenames, on which the test should be conducted. If "all" is specified, then all the nodes in the cluster will be used for verification.

DESCRIPTION:
performs the appropriate checks on all the nodes in the nodelist after setting up Cluster Ready Services(CRS).




Now we run the post-CRS verification with cluvfy to ensure that our Oracle RAC clusterware has been correctly installed. This is useful since the Oracle 10g/11g RAC Clusterware (CRS) is the most challenging aspect of a RAC implementation.


oracle@racnode1> cluvfy stage -post crsinst -n all -verbose

Performing post-checks for cluster services setup

Checking node reachability...

Check: Node reachability from node "racnode1"
Destination Node Reachable?
------------------------------------ ------------------------
racnode1 yes
racnode2 yes

Result: Node reachability check passed from node "racnode1".


Checking user equivalence...

Check: User equivalence for user "oracle"
Node Name Comment
------------------------------------ ------------------------
racnode1 passed
racnode2 passed
Result: User equivalence check passed for user "oracle".

Checking Cluster manager integrity...


Checking CSS daemon...
Node Name Status
------------------------------------ ------------------------
racnode2 running
racnode1 running
Result: Daemon status check passed for "CSS daemon".

Cluster manager integrity check passed.

Checking cluster integrity...

Node Name
------------------------------------
racnode1
racnode2

Cluster integrity check passed


Checking OCR integrity...

Checking the absence of a non-clustered configuration...
All nodes free of non-clustered, local-only configurations.

Uniqueness check for OCR device passed.

Checking the version of OCR...
OCR of correct Version "2" exists.

Checking data integrity of OCR...
Data integrity check for OCR passed.

OCR integrity check passed.

Checking CRS integrity...

Checking daemon liveness...

Check: Liveness for "CRS daemon"
Node Name Running
------------------------------------ ------------------------
racnode2 yes
racnode1 yes
Result: Liveness check passed for "CRS daemon".

Checking daemon liveness...

Check: Liveness for "CSS daemon"
Node Name Running
------------------------------------ ------------------------
racnode2 yes
racnode1 yes
Result: Liveness check passed for "CSS daemon".

Checking daemon liveness...

Check: Liveness for "EVM daemon"
Node Name Running
------------------------------------ ------------------------
racnode2 yes
racnode1 yes
Result: Liveness check passed for "EVM daemon".

Liveness of all the daemons
Node Name CRS daemon CSS daemon EVM daemon
------------ ------------------------ ------------------------ ----------
racnode2 yes yes yes
racnode1 yes yes yes

Checking CRS health...

Check: Health of CRS
Node Name CRS OK?
------------------------------------ ------------------------
racnode2 yes
racnode1 yes
Result: CRS health check passed.

CRS integrity check passed.

Checking node application existence...


Checking existence of VIP node application
Node Name Required Status Comment
------------ ------------------------ ------------------------ ----------
racnode2 yes exists passed
racnode1 yes exists passed
Result: Check passed.

Checking existence of ONS node application
Node Name Required Status Comment
------------ ------------------------ ------------------------ ----------
racnode2 no exists passed
racnode1 no exists passed
Result: Check passed.

Checking existence of GSD node application
Node Name Required Status Comment
------------ ------------------------ ------------------------ ----------
racnode2 no exists passed
racnode1 no exists passed
Result: Check passed.


Post-check for cluster services setup was successful.


The beauty of the Oracle RAC cluster verification tool is that it can pinpoint trouble spots before, during and after the installation process. It is an essential tool in the Oracle RAC expert's workbench.

Some additional details on the use of the cluvfy utility are available in the following Oracle Support online Metalink notes:

ML # 339939.1- Running Cluster Verification Utility to Diagnose Install Problems
ML # 316817.1- CLUSTER VERIFICATION UTILITY FAQ


Regards,
Ben