Basic Cluster Troubleshooting

Posted: June 7, 2012 in Failover Clustering
Tags: , , , , ,

So, just last night, we had an issue with our cluster.

One of our disks would not come online. We were seeing an error event ID: 1304 as below.


So, we began to investigate as all the Technet articles state that the drive is corrupted and Disk Signatures need to be changed, however we have had no major changes in the last little while. We were also able to view to CSV Volume from our SAN management tools. So, we were confused. The disk just would not come online.

1 . The Quorum was online, or else the cluster would not come online.

2. All the nodes were online.

3. All Networks were online

4. The cluster virtual name and IP was up.

However, only CSV would not come online. So, we began to dig a little bit deeper and discovered that the error message was a little misleading, it did however contain some useful information. From the error message we were able to glean the problem node (CSV Owner). We then hopped onto the problem node and after a little digging around, discovered that the volume had been set to OFFLINE. We set the volume to ONLINE and then tried to bring the resource ONLINE in Failover Cluster Manager and hey PRESTO!!! The volume can online and so did my VMs.

I have since created a PowerShell script which can be found here to help the troubleshooting process. All you need to do is provide the list of cluster/s to be checked.  It does the following.

1. Check if the nodes state is in an “UP” state. If the node is in an up state, it then checks the status of the Cluster Service and reports if it is up or down.

2. It then checks the CSV state and specifies the current owner of the CSV and then lets you know if the CSV online or reserved. This is good.

3. Should it get a state other than this, it then proceeds to run a list disk from diskpart on the reported CSV owners to help you find which CSV and which owner to start working on.

Follow me.




Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s