Updated SLD MP

Just noticed that the Service Level Dashboard MP (Solution Accelerator) has been incremented to version 6.0.6278.6. Download at http://www.microsoft.com/downloads/details.aspx?FamilyId=67EF9823-631B-49B7-9D7F-9F125BDF27AE&displaylang=en&displaylang=en

Whats new – “New summary reports based on Dundas Gauges feature visual indicators that enable viewers to quickly grasp how well applications or groups of applications are meeting their service level goals.”

Dundas Gauges – http://www.dundas.com/Gallery/Gauge/RS/index.aspx?ImgGroup=Circular

And what a new report looks like

Very funky.

RMS and Time

I was about to write a post about the RMS going grey when this post came up –

http://blogs.technet.com/smsandmom/archive/2008/08/28/opsmgr-2007-the-health-of-the-root-management-server-is-in-a-gray-not-monitored-state.aspx
OpsMgr 2007: The Health of the Root Management Server is in a Gray “Not Monitored” State

While it describes the same symptoms it is not the same problem. What I was seeing was the RMS health service on a new install going critical every minute and then resolved by the system. That in itself was very strange as the alert was created by a rule and I have always believed that only monitors can be auto resolved. Looks like I was wrong on that.

The alert was “Root Management Server Unavailable” and the description was “The root management server (Healthservice) has stopped heartbeating soon after 8/28/2008 4:10:08 PM. This adversely affects all availability calculation for the entire management group”. There was nothing in the Alert Context. The RMS would flick between grey and green. The auto resolved view I created (very useful view for finding problems that are always coming and going that you may not notice. Also good for checking how baseline monitors are doing if they are getting created and resolved at next time check. – Create a new alert view and chose “resolved by a specific user” and put in System) showed that this alert was happening every minute.

I then noticed that the RMS time was 1 hour ahead so I reset that but in minute (co-incidence? I think not) the server’s time went 1 hour ahead. I tried several wasy of channging the time with the same result. All the other servers were in the same time zone and they were fine. I spent some time with w32tm and net time before checking the VMWare settings and found that it was getting its time from the ESX server. I did not think to look there initially as the build was not supposed to have that switched on. I removed that and the time was picked up correctly from the domain. The ESX server this VM was on was an hour out but all my other VMS must have been on another server as their time was correct.

I had already uninstalled the OpsMgr software to see that it was the OS that was still doing the time shift. Once I reinstalled it with the server at the correct time then everything was OK. Very bizarre.

This is a different problem to the one I had with another system where the AD server and the RMS/DB server were one hour out that caused problems.
http://ianblythmanagement.wordpress.com/2008/08/15/a-brief-history-of-time/

Time is obviously very import to OpsMgr.

Help on Exchange Reports in the Guide

One of the annoying things with doing reports in OpsMgr 2007 is getting the target right so you can get data in the report. I posted about this and Kevin Holman has also done a post. I am pleased to see that in the latest MP Guide for Exchange 2003 (released a few days ago), not only do they tell you what reports will be installed but they have a table showing what the targets are and what you should search for to get those targets. See How To Run Reports near the end of this section of the online version of the MP Guide – http://technet.microsoft.com/en-us/library/cc511014.aspx.

This is a great start and will be of immense use for people who have been frustrated trying to get these reports to show data. Now we just need all the other MP Guides to add similar information.

Complex Monitor

Yesterday I posted about a rule showing how the expressions could be used to build a rule without having to resort to regular expressions. I thought that it was just bit of serendipity and was wondering how complex you could go with this. Just after I posted that I saw another new alert – this time froma monitor called Data Validity Check.

Just look how complex you can go! I always thought that the Formula button was not necessary but with this type of ANDing and ORing you need to see how that will be built up. Quite amazing what hidden deoths that there are in this product.

No More Regular Expressions?

I am working on a project where they want to create a  MP with a lot of rules. So I was looking at regular expressions to see how I could cover a number of events with just one rule. Tricky thing regular expressions. And out of the blue I receiveded an alert that I had not seen before so I thought I would look at it and find out what MP triggered it. Looking at the configuration for the “LearningModule Execution Failure” rule I saw this neat bit of work.

I am not sure if I have read about this before but if I did I can not remember. It seems so nice and simple if you have a block of events that you want to cover. It is amazing what you learn even after using the product for ages.

 

Create Events

Something that is handy for demos and testing systems is to create events in order to check that rules and monitors work and create alerts to order. Usually if you want an event to happen in the event log it never will. As soon as you turn your back thousands happen.

I was confronted with this recently as a customer wanted to do a functional test after installation and want to create events from some of the key MPs to ensure that they worked. Using MP Viewer I can look into the MPs, look at the rules and find out which event numbers with source and event log are in an MP and enabled. Normally for demos I use eventcreate.exe as it is part of the OS. Unfortunately this only does events up to 1000. So I looked around to see what I could find. It was difficult to search for as most searches tend to respond with tools to monitor and manipulate event logs. So there may be other tools out there.

1 Eventcreate.exe

Pros – included in Windows 2003, 2008, XP and Vista; can do remote to another server; can be used in a batch file.

Cons – only up to ID 1000, only Application and System logs

2 Logevent.exe

Pros – can be included in a batch file; can do all event IDs; can do remote.

Cons – Can only do Application log; was part of Windows 2000 Resource Kit but no longer shown in that download area.

3 VBScript

Pros – run from anything that can do VBscript

Cons – Incredibly limited; only one source – WSH; only Application log; event ID equals type of event (0 for info, 1 for warning etc).

4 PowerShell

Pros – Clever use of interactive PowerShell script from Stefan Strange with additions from Ken

Cons – interactive (would need extra work to take parameters to be used in a batch file); can only do local server (but could be extended with more code) and so needs PowerShell installed

5 Event Create (MOM 2005 Resource Kit)
http://technet.microsoft.com/en-us/opsmgr/bb498240.aspx
http://download.microsoft.com/download/d/4/b/d4bfc32c-d8d2-4541-8356-4c6359eecbb4/MOM2005ResKit.msi

Pros – GUI front end that can do remote; any event logs; all IDs

Cons – can not be used in batch file; msi needs .Net v1 to extract its files

6 OpsMgr Script

Pros – simple to create a timed script to run as at a regular interval; can use OpsMgr targeting; could be created as a VB script to run as batch file

Cons – can only log to Operations Manager log; can only use Health Service Script as a source; needs OpsMgr agent in order to work

Example showing how to ping.

Summary

If you need to create events and are happy with the limitations then eventcreate.exe is easy and is included in all the latest OSes. For a demo the Event Create GUI from the MOM 2005 Resource Kit is great. If you can get hold of it logevent can automate event creation (as long as it is the Application event log). Highly recommended is the PowerShell script from Stefan and Ken. With some more work it would be able to do everything.

Updated 26/8/08 – Changed references to logevent.exe as it is not included in MOM Resource Kit as I originally thought. It was in my 2005 VM but when I did a new install only the GUI version is there.

It would be nice to have a single tool like eventcreate.exe but without the limitation on event logs and event IDs. Anyone want to create a tool like that?

Disk Space CPU % Report Part 2

I blogged in June about this nice report that comes with the Virtualisation MP. I received a number of comments and I have seen the post mentioned in the newsgroups. As I said in the previous post it is report that many people want – Product Group take note!

Some info about this report.  I mentioned that the perfmon counters are already collected by the OS MP. I said that because that is what it said in the report description. As Derek points out that is not correct. It seems the people who did this MP were too lazy to update the descriptions. I delved into the MP and it does collect the counters from collection rules that are part of the VMM MP itself. They are collected every 900 secs (15 minutes).  If you search Rules for Virtualization Candidate you will see the eight rules.

The other problem is that is is the virtualisation candidate report. If the server is running on Microsoft virtualistion technologies it is ignored as a candidate. I run the report on servers running under VMWare ESX which means that Microsoft does not consider it a VM! I remember filing a bug in the SCOM beta as there is a column called “Is VM” which only picks up Microsoft VMs. Basically I was told to write my own discovery. It would be interesting to see what they do now given that SC VMM does ESX monitoring as well. The least they can do is change the heading to “Is MS VM”.

Pontus Blomqvist has a post on changing the discovery and this explains why some reports are blank.
http://blopon.blogspot.com/2008/02/why-virtual-candidates-reports-is-empty.html

There is a small issue with the Virtualization Candidates reports.  I have made my own discovery (For HP Servers, since I just have them) and disabled the default as mentioned in the blog post. And now the Virtual Candidates reports works just fine. 

There are a number of discoveries in this MP but the one that counts for this report is the Virtualization Candidate and a class is created that the rules target to collect the data. If the counters are being collected you can look at a view rather than run the report. Under the Virtual Machine Manager view there is an All Performance view but they are not shown there even though that is the MP that collects the data. Create a new view and chose “collected by specific rules”. Scroll down the Select Rules box and near the end are the 8 rules starting Virtualisation Candidate. Select them and you can then see what has been collected for each server. If you have data then the report will run. Otherwise see Pontus’s post.

Assuming that you do not want to use this set of MPs apart from this report then the options are change the discovery or disable the 8 rules and recreate them using a new target like Windows Server so that all servers will be used and not just those that are virtualisation candidates. Or do your own report.

Ziemek Borowski left a comment that you could use the vPerfRAW, vPerfHourly or vPerfDaily views in OperationsManagerDW to create a similar report. There is information on this in the Report Guide but most infrastructure people are not that au fait with SQL queries and SRS. I challenged Ziemek to create a report and he has created a sample report that can be used as the basis for a fuller report.

You can download his MP at http://ziembor.pl/post/SystemCenterReportingPerformanceOverview-MP-0018.aspxboth in MP and XML. The XML makes it easy to see the SQL query.

Here is a PDF of the report when I ran it on my demo system.
systemcenterperformanceoverview

A Brief History of Time

Actually no. But a quick post on how time can throw you in OpsMgr. I recently built a new 2 VM demo system – 1 for AD and the other for OpsMgr (all roles). I fired it up to do some testing on a problem I was working on and I received an alert to say that the AD health service was down. Checked the service on AD. Restarted it. Pinged the RMS by NetBios and FQDN and so on but that alert stubbornly refused to go even though it appeared all was well. I looked at Health Explorer and all the diagnostics that it had ran were green. I was puzzled.

I was looking at the OpsMgr log on the AD server and I noticed an event that was unfamiliar. Event 5500 and words that caught my attention were

“..caused the incoming state change request to be dropped due to it being older than currently recorder state change for this monitor.”

The rest of the message did not make sense but the older made me look at the time on both servers. Both had the same date and time but one was set to GMT, which I always set my servers to as I am in the UK, but for some strange reason the OpsMgr server was set to Pacific Time which is 8 hours behind GMT. That explained it. The AD was heart beating but as it was in GMT time zone the management server saw the heartbeat as being 8 hours old. As soon as I changed the time zone to GMT the alert immediately showed itself in the future so all I have to do now is wait 8 hours for both machines to be back in synch.

Suggestion for the product group. Have an alert when a heartbeat is received that is more than 1 hour out as that will probably be a time zone problem. Also MOM 2005 showed the list of agents and when was their last heartbeat. I tried to see if I could create a view to emulate that but did not find last heartbeat as a property of the agent. I used to find that view useful in 2005 so it would be nice to get in back in 2007.

Now that is sorted all sing along …

“…Let’s do the Time Warp again!…”

New Blog with Good Start

Jimmy Harper works in Kevin Holman’s team and has started a new blog.

http://blogs.technet.com/jimmyharper/default.aspx

His first post shows a great example of creating a three state timed script using WMI to get the state of a service. He has included lots of screen shots and the text of the script making it easy to cut and paste – thanks. He also includes the XML MP if you can’t be bothered typing it in – or cutting and pasting.

Could I suggest a follow up blog of showing how this differs from using the Authoring Template to monitor a Windows Service?

Talking about blogs the new OpsManJam blog is good but although it says it has an RSS feed it was not obvious how to get to it. I finally worked it out but Stefan, who had the same trouble, has posted a guide on how to do it for those of you not familiar with SharePoint.

I met John Rakowski at the recent Windows Management User Group meeting in London and he has a good blog with the great name of MOMski.

Set SPN Warning

You may have seen this alert come up in OpsMgr 2007 SP1 if you used a domain account for the SDK and Configuration service account rather than Local System.

SDK SPN Not Registered

If you click on the View Additional Knowledge it tells you how to fix it.

In the Resolution area of the alert and the Knowledge it tells you the same thing – setspn needs to be run for MSOMSDK. But in the Alert Description it says it is MSOMSDKSVC. This is a mistake that was picked up by a customer and Jeanie Decker has posted that it is going to be fixed.

Use the details in the Alert Description. That is use MSOMSdkSvc. It tells you the correct SPN, the RMS NetBios name, the RMS FQDN and the SDK account to use. All you have to do is get them in the right order after setspn.exe -A.

Update 14th August

Be sure to check out the following posts for more information on this subject.

Jonathan Almquist, Premier Field Engineer, Microsoft
http://blogs.technet.com/jonathanalmquist/archive/2008/08/14/operations-manager-2007-spn-s.aspx

Walter Chomak, Microsoft Consulting Services
http://wchomak.spaces.live.com/blog/cns!F56EFE25599555EC!824.entry

Kevin Holman, Premier Field Engineer, Microsoft
http://blogs.technet.com/kevinholman/archive/2007/12/13/system-center-operations-manager-sdk-service-failed-to-register-an-spn.aspx

I have always prefered to use a domain account for this role but I think I may be using Local System in future.