Operations Manager – Extending UNIX/Linux Monitoring with MP Authoring – Part IV

Introduction

In Part III of this series, I walked through creation of data sources, a discovery, and a rule for discovering dynamically-named log files and implementing an alert-generating rule for log file monitoring.  In this post, I will continue to expand this Management Pack to implement performance collection rules, using WSMan Invoke methods to collect numerical performance data from a shell command. 

Using Shell Commands to Collect Performance Data

Whether it is system performance data from the /proc or /sys file systems, or application performance metrics in other locations, performance data for UNIX and Linux systems can often be found in flat files.   In this example Management Pack, I wanted to demonstrate using a WSMan Invoke module with the script provider to gather a numeric value from a file and publish the data as performance data.   In many cases, this would be slightly more complex than is represented in this example (e.g. if the performance metric value should be the delta between data points in the file over time), but this example should provide the framework for using the contents of a file to drive performance collection rules.   The root of these workflows is a shell command using the cat command to parse the file, which could be piped to grep, awk, and sed to filter for specific lines and columns.  

Additionally, if the performance data (e.g. hardware temperature or fan speed, current application user or connection count) that you are looking for is not stored in a file, but available in the output of a utility command, the same method could be used by using the utility command instead of cat.

Collecting Performance Data from a File

In this example, the MyApp application stores three performance metrics in flat files in the subdirectory ./perf.   I have built three rules that cat these files, and map the values to performance data.  The three rules are functionally identical, so I will only describe one of them.

Performance Collection Rule:  MyApp.Monitoring.Rule.CollectMyAppMem

This rule collects the value from the ./perf/mem file in the application directory, which represents the current memory used by the application in KB.   The rule targets the MyApp.Monitoring.MyApp application class.

Rule Data Source:

The rule uses the MyApp.Monitoring.ShellCommandMonitoring data source, described in Post I of this series, with the configuration:

<Interval>300</Interval>
<TargetSystem>
$Target/Host/Property[Type="MicrosoftUnixLibrary!
Microsoft.Unix.Computer"]/NetworkName$</TargetSystem>
<ShellCommand>
cat $Target/Property[Type="MyApp.Monitoring.MyApp"]
  /InstallPath$/perf/mem
</ShellCommand>
<Timeout>120</Timeout>

Notice that the ShellCommand is our cat command:  cat $Target/Property[Type=”MyApp.Monitoring.MyApp”]/InstallPath$/perf/mem

Rule Condition Detection:

A System.Performance.DataGenericMapper is used as the condition detection module to map the StdOut to performance data, with the configuration:

<ObjectName>
$Target/Property[Type="MyApp.Monitoring.MyApp"]
  /Name$
</ObjectName>
<CounterName>Memory used (KB)</CounterName>
<InstanceName>Total</InstanceName>
<Value>
  $Data///*[local-name()="StdOut"]$
</Value>

Write Actions:

The rule defines two write actions:  Microsoft.SystemCenter.CollectPerformanceData and Microsoft.SystemCenter.DataWarehouse.PublishPerformanceData to collect the data and publish it to the DW.   These require no configuration.

The end result is that the scheduled rule grabs the value from the text file, maps it to performance data, collects and publishes the performance data to the OM and DW databases.  This mechanism can be used for nearly any numerical performance metric that is accessible (in a timely fashion) from a shell command pipeline or script.

Stay tuned for more in this series….