UNIX/Linux MP Authoring – Discovering and Monitoring Failover Clusters

July 30, 2013 Leave a comment

In my last post, I walked through creation of an MP with dynamic discovery for a UNIX/Linux application. In this post, I’ll continue to demonstrate the use of the UNIX/Linux Authoring Library examples for MP authoring, but I will take the demonstration quite a bit deeper into authoring territory – by creating an MP for monitoring of Linux failover clusters. While the base UNIX and Linux operating system Management Packs don’t have built-in detection/monitoring of failover clusters, MP Authoring can be used to build a robust cluster monitoring solution.

In this post, I will walkthrough authoring of a basic MP for monitoring a Linux failover cluster. I have two goals for this post:

  1. Demonstrate the use of the UNIX/Linux Authoring Library for MP Authoring scenarios
  2. Demonstrate the creation of a basic cluster monitoring MP that can be modified to work with other cluster technologies and monitoring requirement

This is fairly involved MP Authoring, and is intended for the author with a bit of experience.

Note:  the example MP described in this blog post (and the VSAE project) can be found in the \ExampleMPs folder of the UNIX/Linux Authoring Library .zip file.

Background

The MP I am building is intended to perform discovery and monitoring of Linux failover clusters, though it could certainly be adapted to work for other cluster technologies.  Prior to starting on the MP implementation, I think it is useful to conceptually model the implementation.

Regardless of the specific technology, failover clusters tend to have the same general concepts.  Entities that I want to represent are:

  • Cluster nodes – hosts that participate in the failover cluster
    • Monitor for requisite daemons
    • Monitor for quorum state
  • Cluster – a “group,” containing the member nodes as well as clustered resources
    • Roll-up monitors describing the total state of the cluster
  • Service – a clustered service, such as a virtual IP or Web server
    • Monitor each service for availability

These conceptual elements will need to be described in Management Pack ClassTypes and corresponding RelationshipTypes.  A basic diagram of my intended implementation looks like:
MPDiagram

Tools and Commands

For both dynamic discovery of the cluster nodes, as well as monitoring of the cluster resource status, I leveraged the clustat utility.

As an example, the clustat output in my test environment, with two nodes and a single virtual IP address as a service, looks like:

[monuser@clnode1 ~]$ sudo clustat
 Cluster Status for hacluster @ Tue Jul 23 19:24:47 2013
 Member Status: Quorate
Member Name ID Status
 ------ ---- ---- ------
 clnode1 1 Online, Local, rgmanager
 clnode2 2 Online, rgmanager
Service Name Owner (Last) State
 ------- ---- ----- ------ -----
 service:IP clnode1 started

As you can see, the output here can be parsed and used in discovery of cluster nodes, cluster, and services, while also providing health information about the cluster nodes, cluster, and services. Depending on versions, clustat may require privileges (i.e. sudo), but it has the advantage of being a status reporting tool that is not used in cluster configuration, so it is a useful command line tool for monitoring purposes.

Creating the MP

Getting Started

Just like with my previous example, starting off on this MP involves creating a new Management Pack project with the VSAE:

GettingStarted

Read more of this post

Advertisement

Filed under Management Pack Authoring, Operations Manager, System Center, UNIX/Linux

UNIX/Linux MP Authoring – Dynamic Discovery of Roles & Applications

July 25, 2013 Leave a comment

In a previous post, I demonstrated the use of Operations Manager’s “Shell Command Templates” to customize monitoring by creating a management pack for monitoring BIND DNS servers. The Shell Command Templates offer a great deal of customization potential through simple wizards, but one limitation is that the templates require the target class to already be defined. In other words, they cannot dynamically discover the presence of an application/role on monitored computers. One way to generate a custom class to represent an application or role is to create a UNIX/Linux Process Monitor with the MP Template and target the Process Monitor to a group containing the UNIX/Linux computers running this service. This will create a custom class representing the process, which can be used as a target for the Shell Command Templates. However, this still requires that the group members are explicitly defined, or dynamically populated based on properties such as the hostname. Such an approach is not a bad option at all, but still not quite dynamic discovery. However, with the UNIX/Linux Authoring Library examples, creating a dynamic discovery of an application or role is a fairly easy undertaking. In this post, I will revisit that prior BIND DNS monitoring walkthrough, but demonstrate how the UNIX/Linux Authoring Library can be used to dynamically discover the DNS servers.

Note:  the example MP described in this blog post (and the VSAE project) can be found in the \ExampleMPs folder of the UNIX/Linux Authoring Library .zip file.

Walkthrough:  Dynamic Discovery for UNIX/Linux, using the Authoring Library examples

I will use the Visual Studio Authoring Extensions for this project, so starting the project entails creating a new MP project:

NewVSAEProject

Following the direction in the Getting Started documentation for the UNIX/Linux Authoring Library, I will then set up MP references and aliases. The suggested references in the Getting Started documentation are mostly sufficient, but I will add Microsoft.Linux.Library.mp (with an alias of Linux) as well in order to target discoveries to the Microsoft.Linux.Computer class.

Management Pack ID Alias
Microsoft.Linux.Library Linux
Microsoft.SystemCenter.DataWarehouse.Library  SCDW
Microsoft.SystemCenter.Library  SC
Microsoft.SystemCenter.WSManagement.Library  WSM
Microsoft.Unix.Library  Unix
Microsoft.Windows.Library  Windows
System.Health.Library  Health
System.Library  System
System.Performance.Library  Perf
Unix.Authoring.Library  UnixAuth

Discovering BIND DNS Servers

I’ll use a basic shell command to detect the installation of BIND on Linux computers in order to dynamically discover it.  This method could be used in multiple ways, such as:

  • Check for the presence of a file or directory (such as an installation directory, init script in /etc/init.d, or configuration file)
  • Check for an installed package with a package manager like rpm
  • Check for a running process with ps

In this case, I’ll look for a config file to indicate that BIND is installed on the computer.  Depending on the UNIX/Linux distro, and version of the application, the artifacts on the system that indicate the installation of the application may vary.  In my example, with BIND 9 on CentOS, I will look for the existence of /etc/named.conf to indicate that BIND is installed. The command I will use is: ls /etc/named.conf |wc –l  This will return a value of 1 if the file exists, and 0 if it does not.

To add the dynamic discovery to the MP, I will add a new Empty Management Pack Fragment to the VS project, named Discoveries.mpx.

mpfragment

Read more of this post

Filed under Management Pack Authoring, Operations Manager, UNIX/Linux