Oracle RAC Blog

RAC/CRS Stack will not start after host reboot.Problem, Analysis, Resolution

July 7, 2009 · Leave a Comment

In two node RAC environment, the UNIX hosts reboots are known to cause variety problems
for CRS stack. Usually the first node comes up clean and the second one will start
 writing messages to all the evm, client, crs logs, a very conflicting and confusing messages.
There are myriad ways of adressing the issue as mentioned in OTN, and other tech forums based
on same type of error messages. Nevertheless, none of the solutions have worked for us.
 While one can spend a day in creating an SR and wait
 for another week to resolve, thought I would share this troubleshooting experience that
 saves fellow RAC-ites some time and energy with a similar kind of issue.

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
If you see the below type of error message>>
–[ COMMCRS][1]clsc_connect: (1002f4fe0)
–[    EVMD][1] EVMD waiting for CSS to be ready err = 3
–[ CRSRTI][1] CSS is not ready. Received status 3 from CSS. Waiting for good status ..
– Voting disk offline
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
We have addressed the problem by understanding that the CRS is unable to start,
in absence of no symptoms of OCR, voting disk corruptions. Additionally,
 the evmd daemon is waiting for css to come up and seems to have hung.
What we have noticed is, while bring up the CRS stack deamons, Oracle writes
 the socket files to /var/tmp/.oracle directory. This directory should be
clean in order for CRS to come up. Cleaned up existing socket files, rebooted the node2.
All RAC components started working without any issues.
We have scrapped the SR draft for Oracle and the resolve to resolve the CRS issue paid off.
 
Hope the troubleshooting tip would be useful…

Categories: Uncategorized
Tagged: , , , , ,

0 responses so far ↓

  • There are no comments yet...Kick things off by filling out the form below.

Leave a Comment