Exchange White Space

This is something that Exchange admins ask for. The trouble is that the number is buried in the event description.

SCOM has an event rule already that collects these events. The rule is called “Free space available in databases after online defragmentation.” and the event ID is 1221 with source “MSExchangeIS Public Store” from the application event log. You get a sentence in the event description like

“The database “XXXMSX02 SG1\XXXMSX02 SG1PF1″ has 63 megabytes of free space after online defragmentation has terminated.”

What I did was create an event view for that ID and the admin can just quickly go down the list for that day and look at the numbers in the description. People that know scripting may be able to do something funky to collect the information and alert on it depending on the size of the number. But I will leave that for someone else to do. The Exchange admin was happy with the event view as it collated all the information in one place and made it easy to see the details.

It would be better if the Exchange dumped that into a performance counter or WMI so that it can be look at and alerts created.

Odd Agent Behaviour

At a few customer sites I have seen some odd issues with the agent. I thought I would write them up if anyone else has these problems.

Agent (SP1 RC) stops PowerTerm service

This is the first time I have ever seen a MOM or SCOM agent have this effect but when the agent is installed the PowerTerm Terminal Server Agent Service stops. No warnings or any clues in event log. Tested on a dev server as well and the behaviour is the same. The issue has been raised with Ericom. Did not have a chance to try it with the final SP1 agent.

Appsense

This product seems to block scripts like McAfee but it is just a configuration issue that is need to allow scripts to run.

SP1 Agent problem with false directory

A very weird one. The agent installed OK but went grey and the inventory never got updated so you could not see the version number of the agent. There were WMI probe errors from the server and one about the event logs and all seem to indicate that it was a rights issue.

Module was unable to connect to namespace ‘\\FQDN\ROOT\CIMV2′ This has happened 1 times since this instance was loaded. HRESULT: 0×80070005 Details: Access is denied.

The Windows Event Log Provider was unable to open the Application event log on computer ‘fqdn.co.uk’ for reading. The provider will retry opening the log every 30 seconds. Most recent error details: Access is denied. One or more workflows were affected by this. Workflow name: many Instance name: many Instance ID: many Management group: SCOM2007

The OpsMgr event log was red with failed discoveries.

I restarted the service and reinstalled the agent with no success. But there was a DCOM error 10,000 after installation and looking through the system log this had happened a few times.  Also the local Exchange person said that they had tried to install a product onto it and had DCOM issues. Searching on this showed a problem with Terminal Services that needed a registry change. However when we actually logged into the server there was an error message saying that c:\program existed and it may affect c:\programs files. This was a zero bye file called program. When this was removed the agent started working correctly again. Weird.

Health Roll Up Shows Warning when all Unit Monitors are Green

This was with the SP1 agent. We found this post http://www.expta.com/2008/02/fix-for-scom-aggregate-health-state.html which showed that putting it into maintenance mode for 5 mins cleared it up. That worked but obviously not fixed in SP1. Consequently I saw Cameron’s post that says about using the Flush Health Service State task. Worth a try if I see it again.

Veritas Storage Foundation MP

Veritas Storage Foundation Version 5.0.1000.0

MP available at http://seer.entsupport.symantec.com/docs/288628.htm
MP Guide (69 page PDF) available at http://seer.entsupport.symantec.com/docs/288629.htm

Although this is a sealed 2007 MP it is pretty obvious that it has been converted from 2005. Reliance on Backward Compatability, lots of rules rather than monitors, the knowledge is still in MOM 2005 format and the guide still taks about the MOM 2005 alert levels rather than the 2007 ones. This is amusing as they say on the download page “Symantec does not recommend upgrading an existing SFW MOM pack with the Microsoft conversion feature.”
The MP page says it is for VSF v5.0 but when you read the guide it supports v4.3 as well.
“This version of the SFW Management Pack is supported on SFW 4.3.x, SFW 5.0, and later versions running on Windows Server 2003.”

The guide does do a good job of listing the rules, whether they are enabled or not and what level of alert they generate. I just wish more MP guides went into this level of detail.

Note that the 2005 MP for VSF v4.3 is in the catalogue but the link is broken.

Free Engyro Connectors

A number of people have mentioned these are now going to be free. And the general feeling is that this is a good thing. Well free is good isn’t it? I have misgivings. Let me explain with a bit of history.

In the days of MOM 2000 organisations were interested in the product, especially in its ability to monitor Exchange and AD which has always been MOM’s strong point. They generally would have another system like Tivoli for example. The only way that they would consider using MOM was if it could connect to their manager of managers. Skywire’s iWave was a leader in this field in those days. But then the objection came up that if they used a 3rd party connector who do they go to when there is a problem. So they wanted a connector that was either from Microsoft or their manager of managers company. Well IBM (Tivoli), HP (Openview), BMC (Patrol) or CA (Unicenter) had no desire to let Microsoft steal part of the estate that they were managing. So it was up to Microsoft. Except that the product group were less than enamoured by this prospect as they did not want to be tied to the other companies release cycles. With a lot of persuasion (especially from Sally Baldauf, Microsoft NY) they decided that they would do something.

What the product group did was to pay Engyro to make the connectors for them and they then released those connectors (Tivoli and HPOV) as resource kit tools so that they did not have to support them. They were released as free tools. The problem with that strategy was that customers that used them and found issues or needed extra features could not get them. They had to go to Engyro for support and buy a newer version from them if they wanted the latest features.

That is what worries me about making the Engyro connectors free. I would like to have Microsoft give an assurance that not only will they support these connectors but they will be upgraded and new connectors added. The trouble is that if they do not do this there is no longer Engyro to fall back on as there was with the original connectors.

I also think that the timing is interesting. I did speculate that in an earlier post that the reason that Engyro was purchased was more for Service Manager rather than Operations Manager. And as they have recently announced that Service Manager will be delayed until 2010 then it makes sense to release these connectors to get them used. It would be interesting to find out what happened out Engyro’s UNIX monitoring. Whether it has been shelved totally or sold off to someone else.

The other problem I have with the free connectors is that it puts partners (like Skywire and eXc) at a disadvantage. They have invested in building connectors and the majority will go the free Microsoft route which gives then less money to invest in other connectors. If the Microsoft connectors had a price then it would spur organisations to do a comparison. And if there was a price it would help convince me that Microsoft connectors would be supported and enhanced.

So Engyro connectors being free – good or bad? Only time will tell.

Exchange MP Roadmap

From the newsgroup by Åke Pettersson [MSFT].

“Hello, I wanted to update you on the roadmap for the Exchange MPs on OpsMgr.

Exchange 2003
There is an updated Exchange MP on the OpsMgr SP1 CD. There will also be an updated MP guide to cover the fixes in this release. The MP and MP guide will be on the MP catalog, they are about to be released to the MP catalog.
There will be another update to the Exchange 2003 MP during March. We will update the MP guide with the fixlist for this and do a general overhaul of the MP guide to make it consistent with the MP.

Exchange 2007
There will be an update to this MP during Q2. It will be a converted MP focusing on high-priority improvements such as Exchange 2007 SP1 support. At a minimum we will also do release notes for this MP to cover the fixlist.”

Too Many Trusts

At a customer site they had not loaded SQL 2005 on the server so I said that I would do it as part of the install. However when it got to “Installing Local Groups” it just seemed to freeze. I tried a few different accounts and the same thing happened. And when cancelled it was taking for ever so I ended up rebooting before doing another install. A quick search showed that others had the same problem but when left it finished. Eventually found a KB article (910070) that explained that the setup program was trying to lookups and the time taken increases dramatically. This organisation had over 70 external trusts. This KB mentions a hotfix that you can request but then you have to slipstream it. There is another KB (818024) that mentions a registry setting to help when there are too many external trusts.

As time was limited I took the easy option for disconnecting the network! This meant having to do the install in a noisy computer room instead of remotely. Even then as we were using a domain user for the service account we had to re-enable the NIC for that look up and then switch it off again when it “froze”. Once the NIC was disabled it quickly bypassed the “Installing Local Groups”. It was fiddly as we had to do it three times during the install but better than waiting for it to do its checking. Weird setup program.

This problem reared its head again as the AD MP has a script to check trusts. This runs every 5 minutes with a timeout of 120 seconds. So we were seeing a number of script errors from the DCs. As a recommendation I change the script errors from warning to information (and did so on 2005 as well). A script that has a one off fail is not important as it could be that the server was busy. Only if the repeat count goes up is it important. Well these repeat counts were going up – every 5 mins and it was the trust script timing out.

The nice thing about 2007 is that it runs standard cscript. And from the alert you can see what the script is and the directory of where the script is located. Just navigate to that directory, drop to a command prompt (I always recommend using Cmdhere from the resource kit) and run the script as per the alert (with parameters as shown) and see what happens. In this case the script did finish but it took a while. A quick search for the monitor running the script and I created an override that increased the timeout to what we had seen. I also had to increase the time of the script being ran from 5 to 20 minutes. But as the customer said as they had not had a tool to monitor the trusts before even doing it once a day would be a big improvement.

I suspect most of you will not encounter that many external trusts but if you do you now know the answer.

Non Catalogue MPs

Although the best place to find new MPs is in the catalogue there are MPs done by vendors whose MP is only on their site and not in the catalogue. And there are a number of community MPs that people may find useful.

As WordPress has the ability to create additional pages beside my blog I have created a page dedicated to these MPs. Mainly because it makes my job easier to find them when I need them but if I find it useful I am sure others must as well.

http://ianblythmanagement.wordpress.com/mps/ 

Systemcenterforum.org has a number of these but they are mixed in with other stuff and I am finding it difficult to find them quickly. (Need a facelift Pete – for the site I meant )

I am focusing just on 2007 MPs and ones that are not mentioned in the catalogue. Drop me a line (ianblyth at gmail dot com) if there is one that you think should be on the list.

SCOM Authoring Console First Looks

A few people now have mentioned that this has been released. It is RC and needs SCOM SP1 I believe. It can be downloaded from http://www.microsoft.com/downloads/details.aspx?FamilyID=6c8911c3-c495-4a03-96df-9731c37aa6d7&DisplayLang=en and there is a 32 bit and 64 bit version. 

Documentation to help with creating MPs :

Authoring Guide (now a whopping 238 pages long compared to 118 for the earlier edition).
Operations Manager 2007 Management Pack Authoring Guide: This document provides detailed information about how to create a Management Pack for a product. The product can be an application, a service, or a device.
February 22, 2008
Download the Guide

Report Authoring Guide
Provides scenarios for custom report creation and reference information on report parameters.
February 26, 2008
Download the Guide

Best Practices for Targeting Rules and Monitors in Operations Manager 2007
The poster illustrates some of the most common scenarios and provides best practices on how to properly target rules and monitors in System Center Operations Manager 2007.
November 19, 2007
Download the poster

System Center Operations Manager 2007 SDK
This software development kit (SDK) documentation includes descriptions and examples that show how to automate and extend Operations Manager features.
2007
Read this online

Also remember http://www.authormps.com/dnn/ as a great resource on learning how to create MPs from scratch with a simple editor. (You need to create a login to access the tutorials.)

I installed the 32 bit version on my demo system and it was a simple install. The program unzipped 3 files and then ran the setup. One tick to confirm the EULA and I was done.

The console itself looks like OpsMgr.

I remember seeing a demo of an earlier version at last years MMS and the guy I was chatting to afterwards summed it up when he said that he thought he was getting on top of OpsMgr until he went to that session!

I have no idea how to create an MP yet – 238 pages is a lot to read and the console is full of jargon about classes, relationships and service models and I am still playing with it. It comes with one template when you create a new MP. I presume that more will be released? I need to rewatch that MMS session! But just trying to create one and play with the options it looks like it will need a whole lot of training to get good with it.

But at first looks it is a great tool to investigate an MP (either MP or XML format) and see what rules, monitors, classes, discoveries, views etc are in it and the properties of these elements. Although you can search for a lot of this stuff in the OpsMgr console this allows you to load an MP at a time and investigate it. You can not save or save as if it is sealed but you can with unsealed MPs. Worth a look just to investigate MPs before you import them even if you don’t create any.

Scheduled Task MP

This MP has been released by MVP Maarten Goet, Inovativ and it is worth looking at even if you do not think you are using scheduled tasks on your Windows servers. I installed it at one company as they had one server that did a number of scheduled tasks and they wanted that monitoring. What they found was a large number of tasks on a number of servers. Some of those had not been run for a while and some failed as the account used for the task had a different password now. So a bit of housekeeping was in order.

I am impressed as I have been looking at how to do MP discovery and it is a lot more complex than just picking up a registry key and creating a group as in 2005!

The MP uses schtasks.exe which means it will not run on Windows 2000. Although the document says it will run on XP, Vista, 2003 and 2008 it does not explicitly mention that it does not run on 2000. So if you have 2000 servers then be sure to exclude them from the discovery rule otherwise you will constantly get script errors. If you want it to run on Windows 2000 there are a few web sites that show that by using a hex editor to change one location it can be run with Windows 2000. Totally unsupported by Microsoft I would suspect. Here are 2 examples.

http://www.jsifaq.com/SF/Tips/Tip.aspx?id=5335

http://www.windowsitpro.com/article/articleid/25186/how-can-i-use-windows-xps-schtasksexe-tool-under-windows-2000.html

There is an object discovery and a monitor that fires off a script. Both run every 60 seconds with a 20 seconds timeout. This is a bit aggressive and I saw lot of script errors as it hit busy servers. These timings are great for demos but unless you really need to know quickly I suggest changing the monitors to run every 3600 seconds (1 hour) or longer for the discovery. If you need to know the state of the job use eventcreate.exe to create events at errors in the batch file run by the tasks and create rules to monitor those. I also created an override for the 20 second script timeout on the discovery rule but there was not an override option for the monitor.

There are two views created. The first is a general overview listed by server.

The second shows the list of tasks from all servers and although the server name is in the next column the task name comes up as the source so you may want to think about a standard naming convention across all the servers! Have a task just called Monday on 50 servers may not be a good thing!

You can also do Health Explorer to find the broken task.

This is a great little MP with the caveats of not doing Windows 2000 and the rules/monitors running too frequently. Most of this can be changed with overrides. Highly recommended.