Posts Tagged ‘Runbook’

So, here is the scenario, we managed a lot of customer’s System Centre Operations Manager (SCOM) environments. One of the most common issues we run into, it is the “Grey Agent” issue, where an agent is no longer reporting into SCOM. There might be a few reasons for this, however one of the most common and effective ways to fix this to clear the agent cache. By this, we simply mean connecting to the agent, stopping the “SCOM” service, deleting the content of the “Health Service State” folder and then restarting the “SCOM” service.

Yes, this is a perfect candidate for PowerShell and their a quite a few scripts that do this in numerous ways using PowerShell, I have a script for this, but they are usually dependant upon a list of some and then loop through this, I decided to use my friend, System Centre Orchestrator (SCO) to facilitate this is in a time manner, with more flexibility and log building as well as inputting the results into a database. With SCO, we also have more avenues available to us for error handling, like logging a call within SCSM or “richer” email or the like.

So, I have learnt with SCO, the best thing to do is to actually sit down and whiteboard you solution, simply draw out the steps you want to follow and think of some error handling. With my example, my logic was as follows. I have added a VIsio diagram as my handwriting is barely legible even to me 🙂

1. Query Database for grey agents, there is a SQL Script for this.

2. Create Folder for logging

3. Read SQL results into a file for “’looping”

4. DNS Test

5. Ping Test

6. Determine Service Name and folder path (Remember we might be dealing with multiple versions of SCOM here

7. Check Service status, to determine if a stop of the service is needed

8. Stop if needed

9. Delete files

10. Wait 10 Seconds

11. Start Service

12. Write log to Database

SCOM_GreyAgentFix

The SQL query will be part of the Runbook file, it can be found here, please change the extension to .ois_export.

Have fun automating.

(E-Mail me)

Follow me,

Twitter (Personal & System Centre)

Twitter (System Centre Focused)

Just recently in a test environment, a script or some such other gremlin caused some absolute havoc on my system. Before you ask, I had NO Backup in place of this system. This system was System Center Orchestrator. However, once a week, I do a “sort-of” backup. By this, I mean I export all my Runbooks and place a copy on my local machine and a file server and the SCORCH (System Center ORCHestrator) server.

I was planning to perform an upgrade to System Center 2012 Service Pack 1, and as it happens according to the Upgrade Sequencing for System Center 2012 SP1, SCORCH was the first Product that needed to be updated.

So, at this moment in time, I had a non-functioning SCORCH server and a backup of the Runbooks from the server WITHOUT a proper backup. Luckily the use of SCORCH at this moment in time was mainly for timed tasks and Runbooks triggered by folder changes.

So, I had to rebuild the server. These are the steps I followed and it worked out for me.

NB!!!! Please note that is NOT a replacement for a backup, having a proper backup plan is crucial to any environment. Once my SCORCH server was back up and running, I immediately implemented a proper backup procedure using Microsoft System Center Data Protection Manager in addition to the backup of Runbooks. I am currently working on a Runbook to backup my Runbooks, more to follow on that soon. To achieve this goal, I will be using an Orchestrator Codeplex runbook.

Anyways, back on topic. Lets “recover” System Center Orchestrator 2012.

1. Reset the computer account in Active Directory.

2. Re-join machine to domain using the same name.

3. Install SQL Server, I used a local instance for this.

4. Install System Center Orchestrator 2012 Service Pack 1. I decided to upgrade as well as part of the rebuild, previous version of Integration Packs are backwards compatible.

5. When installing, make sure to use the same port numbers and user accounts as previously used when installing (this should be documented as part of the original install)

6. Import Runbooks.

SNAGHTML377cebf

7. Once the Runbooks are imported. You will then need to check the Runbooks. You will need to re-register and re-deploy the integration packs.

8. Once all the Runbooks are registered and deployed, your environment will be backup and running.

9. If you have a Connector within System Center Service Manager, you will need to check and ensure that the Run As account is working as expected.

(E-Mail me)

image_thumb_thumb_thumb_thumb

Follow me.

facebook-small322252222 Facebook (Personal)

twitter-small322252222Twitter (Personal & System Center)

scsmlogo2523 Twitter (System Center Focused)

MCC11_Logo_Horizontal_2-color_thumb_

So, just recently I decided to install System Centre Orchestrator. It appears easy enough if you follow the Technet articles. This addresses the need for Service Accounts and the permissions needed by these accounts and there functions. It also helps you to size and decide the Orchestrator installation type that would best suit your environment. It helps you to decide the best way to break the components if you want to.

However, the one thing that I found to be lacking was the actual components needed by SCORCH, like the Features and Roles. I followed the default Technet articles and setup the required accounts and then proceeded with the installation of SCORCH, one of the easiest installs. However, one component would not install, the Orchestrator Console and this was somewhat annoying and would not do. I did some searching around and found quite a handy PowerShell script which was used by someone else who ran into a similar issue. This script enables a few components and once this was done, the SCORCH installation completed without issue. See the script below.

Add-WindowsFeature Web-Static-Content,Web-Default-Doc,Web-Dir-Browsing,Web-Http-Errors,Web-Http-Logging,Web-Request-Monitor,Web-Filtering,Web-Stat-Compression,RDC, NET-Framework, NET-Framework

This worked a treat, so now I was one step closer to Orchestration. So, I designed a few simple Runbooks for testing purposes. I then kicked off the required Service Requests within Service Manager and would then try to track the progress of the Runbook, only to discover that the Runbooks were not running at all. It was NOT a permissions issue or an access issue. After much troubleshooting, I discovered it was something very simple. The services were not started, mine did not appear to start automatically when installing SCORCH.

Just a simple post to make someone else’s life a little easier when installing SCORCH.

Follow me.

facebook-small32225222 twitter-small32225222

MCC11_Logo_Horizontal_2-color_thumb_[2]