Using the Operations Manager 2007 R2 Workflow Analyzer

I’ve only had my hands on the OpsMgr MP Authoring Resource Kit for about 24 hours now, but already the tools are proving to be invaluable.   This post describes a problem that I was able to investigate with the Workflow Analyzer tool to determine the exact cause of the issue.

Background

In a management pack I’m working on, I had a composite workflow designed to calculate SNMP network interface throughput and utilization by collecting the 32bit and 64bit in and out octet counters for an interface.  The SnmpProbe passes the values for all four VarBinds to an Expression Filter, which confirms that either VarBinds 1 and 2 (64bit) or VarBinds 3 and 4 (32bit) have values greater or equal to zero.   The Expression Filter than passes matched data items to a PowerShell property bag probe, which compares the values to a previously collected value set (stored in a temporary file in the file system) in order to calculate delta values and interface utilization and throughput.  

The script was written to use the 64bit counters if data are returned for the 64bit counters and 32bit counters if 64bit counter data are not returned.   I had been having some issues with this workflow when targeted to interfaces of devices that do not support 64bit interface octet counters.  From the lack of errors in the log, and evidence that the PowerShell script probe was not running (no temporary file being generated for these instances), I had concluded that the workflow was stopping with the post-SnmpProbe Expression Filter, but I didn’t know exactly why.   I had thought the Expression Filter was configured in such a way as to continue even if null values were returned for the 64bit counters. 

Using the recently released Operations Manager 2007 R2 Workflow Analyzer, I was able to drill into the actual processing of the workflow and identify the issue.

Workflow Tracing

The steps I used to debug this workflow were:

Launch the Workflow Analyzer and create a new session:

Using the filter, identify the workflow to trace:

Right-click the workflow and choose trace, then identify and select the specific instance to trace, and click Start:

The trace will start and show step by step activity for the workflow.   The trace will first show the trace overrides being set and the workflow module initializations.  You will have to wait for the next scheduled interval to see the actual workflow steps.  Once the workflow begins processing, you can see the results for each module.  

By analyzing the results of the Expression Filter modules, I was able to confirm that the workflow was stopping when it analyzed the Expression Filter after the SnmpProbe, but the issue was a result of being unable to resolve SnmpVarBinds[4].  

By stepping back to the actual received data item (and double clicking it), I could see the SNMP DataItem returned by the SNMP Probe.  It became quickly apparent that only three SnmpVarBinds had been returned.  After double-checking the SnmpProbe module configuration and OID strings, I realized that the SnmpProbe appears to stop processing SnmpVarBinds once it hits an OID for which a value wasn’t properly returned (the first of the 64 bit counters).  The syntax for this returned SnmpVarBind,129, translates to an: SNMP_SYNTAX_NOSUCHINSTANCE error.

This meant that the fourth SnmpVarBind wasn’t available for processing, and having only three SnmpVarBinds in the DataItem throws off the Expression Filter that expects either VarBinds 1&2 or VarBinds 3&4 to be greater than zero.  In this case, VarBind 1 is null and VarBind 4 is missing, resulting in both an error when resolving VarBind 4 and preventing the Expression Filter from finding a match.   Of course, even if the Expression Filter were modified or removed, the workflow would still have problems because it would be unable to resolve all of the SnmpVarBind variables passed to the script probe.

In the end, I opted for a different tactic for this workflow.  I defined overridable data source parameters for the InOctets and OutOctets Object Identifier strings, and I added a “Supports 64 Bit Counters” property to the Network Interface object (and a discovery to discover this property).   This property is then used to populate a group for interfaces that support 64bit counters, and the group is then used as an override target for overrides that set the In and Out octet OIDs to those for the 64bit counters.