Dealing With New MP Alerts
There is a thread on the OpsMgr forum about the pros and cons of having the MPs directly imported in from the web catalogue in R2. Marnix Wolf has a good post about it that summarises the issue. One of the key ones being that it bypasses the manual which is needed in most cases to know how to configure the MP – RTFM. It is useful for me to download known MPs when I create a new installation but for a production environment change control is essential.
One of the other things that happens if you do not test it is that you could get flooded with alerts as I found out recently when importing the DFS MP at a customer site. Luckily I was not the one that received the 600 e-mails from the alerts created. In a new installation the agents are pushed out a few at a time and an MP is added and then tuned. So it is easier for new installation plus it is not in production yet so no-one is looking at the console and subscriptions have not been setup for e-mail alerts. If you have a system where all the agents are already out and it is in production then you need to be careful before importing a new MP. Here are some suggestions that I have developed or picked up from others that should be used to ensure a new MP does not create a flood of alerts.
Method 1 – Don’t Import
Just because you can import an MP does not mean that you should. If you are not interested in monitoring a particular area then why import the MP? I always say to customers that the only alerts that you should see in the console are actionable alerts otherwise the console gets cluttered as people rarely keep it tidy and the Ops guys start to discount the console as they see too many alerts which don’t mean anything to them.
Method 2 – Use Silect
Silect have been making MP Studio for years. It allows you to test an MP without actually deploying to tell you about what alerts would have been created. It does other stuff as well like tracking changes but the problem is that it does cost money which puts some customers off. But if you have invested in Operations Manager rather than one of the other expensive monitoring frameworks then you should have some spare money to buy this.
Method 3 – Create an override MP in test
This method assumes that you have a test environment. Even a single OpsMgr server in a virtual environment would do. With many versions of virtualisation being free (Virtual PC, Virtual Server, Hyper-V Server, VMWare Server, VMware ESXi and Citrix XenServer amongst others) and a 180 evaluation copy of OpsMgr being available most organisations should be able to set up a single server test.
OpsMgr allows you to multi home up to 4 management groups. When the agent is first deployed to a server it does the actual install. When you “install” the 2nd, 3rd and 4th agent it does not install another agent but creates new registry keys and keeps the rules from one management group separate from another. This means that you can push the agent out from your test server to servers that have the application of the MP you are testing and you can find out what alerts are created without those alerts (and potential subscription e-mails) going into the production console.
Now you can create an override MP for those alerts and when you import the MP into production you import your override MP in at the same time so that the MP is already tuned for production. You can then remove the agents from the test management groups until you need to do another test. This may not be feasible for some environments with strong change control as even deploying those reg keys as a 2nd agent is seen as a change that needs to go through the process.
Method 4 – Disable discoveries
Most MPs start discovery as soon as they are imported and so rules and monitors go to the agents and start running and creating alerts. New MPs like the Exchange 2007 MP for R2 are different. It only has a lightweight discovery that discovers the components. If you want these components actually managed then you have to switch them on which means that you can control the alerts coming out. I hope all future MPS use this method.
You can import the MP in and quickly try and do an override to disable discoveries but you may not be fast enough. Use a test environment like above and then import the MP into that. Create an override MP and disable discoveries. Created a group in this MP and switch on discoveries for this group. Now import the MP and override MP into production. You can now add agents one at a time into the group so that only a subset of servers are running the MP. This should help control the number of alerts that you see. As a new “agent” is not put on production servers then this is one less change control to do.
A couple of tools that you can use to help with this are Silect MP Studio Lite. This is a free cot down version that allows you to examine an MP file so you can see what discoveries, rules and monitors are in the MP. Boris Yanushpolsky from the Product Group created MP Viewer. The latest is version 1.7. This allows you to point to an XML or MP (without having to convert it to XML first) file and open it up to view the contents in a structured manner. Stefan Stranger keeps a good list of OpsMgr tools.
Examine the MP and decide whether or not you actually want to import it.
Read the MP Guide before importing the MP. You will find out what needs to be configured.
Use a tool to examine an MP before importing it. You will get an idea of the number of rules and monitors as well as discoveries in the MP.
Use one of the methods to above to reduce potential alerts before importing the MP. If you have the money Silect is a good way to go but otherwise method 3 is best as you are actually doing the tuning without impacting production.
Remember tuning is an ongoing activity and not a one off. You should have a process for it.