SCOM: Locally Monitoring a Listening TCP Port

Typically, one would monitor a TCP port remotely, from a designated watcher node, as a means of confirming availability of a network service, but in some cases, this may not be the most desirable method to poll for TCP port status.   For example, if you wanted to monitor the availability of tcp port on a large number of servers that are SCOM agent-managed systems, where you are concerned with the availability of the particular port only when the rest of the system is functioning normally, it may make more sense to monitor this port status locally from the agent.   This minimizes the number of false alarms (each time the monitored nodes are taken down for maintenance, the remote monitor will throw alerts) and also makes deployment much easier (simply target groups for overrides to enable the monitor).  
A simple VBS script that calls portqry.exe (from the Windows Support Tools) or “netstat –an” and parses the output to confirm port listening status can fulfill the monitoring role in this scenario.   I wrote such a script that will use netstat –an to check the TCP ports currently in a listening state on the localsystem  and parse the output to determine that the defined TCP port is in a listening state. 

The core logic can be seen in this excerpt:

sCmd = “netstat -an -proto TCP”
set objShell = createobject(“wscript.shell”)
set objExec = objShell.exec(sCmd)
set oStdOut = objExec.stdout

bl_Healthy = false

Do until oStdOut.AtEndofStream
 sLine = “”
 sLine = oStdOut.ReadLine
 if instr(sLine, “LISTENING”) > 0 and instr(sLine,”:” & nPortToCheck) then
  bl_healthy = true
 end if  
loop

The full script can be downloaded here (this is provided as-is, with no guarantee of function or support, test before deploying, etc).

Walkthrough on creating a corresponding Unit Monitor behind the cut

Read more of this post

Watch Those Variant Types

When creating some types of Unit Monitors (such as SNMP polls) in the OpsMgr GUI, SCOM will set the default type of the values in the expressions to be strings.   If your health monitoring logic performs numerical comparisons, this will create unexpected and unreliable results, because the monitor will be performing comparisons on the numbers as strings, instead of as numerical data (if the system believes the values are strings, greater than and less than operators are concerned with alphabetical order, and not numerical comparisions).

The good news is that the fix is rather easy, just export the management pack, edit the type definitions in the XML file, and import it back.

For more details:

Read more of this post

“Must Have” Tools – Terminals

This utility has so changed the way I work, it’s hard to imagine life before it.  Terminals is a remote desktop utility that implements tabbed windows for RDP, VNC, SSH, ICA (and more) remote connections. 

Get it here: http://www.codeplex.com/Terminals

“Must Have” SCOM Monitors – Long Running SQL Block Detection

Marios Philippopoulos did a great job  with an article on monitoring long-running SQL blocks with SCOM 2007 at SQLServerCentral.com.    The article:  Monitoring Database Blocking Through SCOM 2007 Custom Rules and Alerts (http://www.sqlservercentral.com/articles/Blocking/64038/)  provides a set of scripts for detecting (and display vital information about) long-running blocks in SQL Server 2000 and 2005 as well as a walkthrough on configuring the SCOM elements for alerting.  This is an absolute life-saver and can really cut down on troubleshooting time when dealing with application issues caused by problematic SQL blocking.

The only customizations I’ve made to this monitoring configuration when implementing are some changes to the SQL script to tune the block duration threshold as well as increase the size of some of the type definitions in the temp table to allow for longer command name output.

I’d highly recommend this article to anyone using SCOM to monitor SQL Server.