SCOM: Advanced SNMP Monitoring Part III: The Completed Cisco Management Pack
September 14, 2009 63 Comments
Note: This management pack has been replaced by the more feature-rich xSNMP suite.
I still have some more full-scale testing to complete, but I have finished the first version of the Cisco monitoring management pack that I have described in the last few posts. This version is R2 only, but I hope to have an SP1 version of it complete soon.
I have decided to license this management pack under the GNU Public License; so in short, you are free to use, modify or distribute it as long as any distribution maintains the original attribution. Be sure to check back at this site for updates to the management pack, and feel free to let me know if you come upon any issues with this management pack or want to request additional features.
More complete documentation of the management pack can be found in the zip file, but the management pack elements are as follows:
Classes
- Cisco Device
- Cisco Interface
- Cisco Power Supply
- Cisco Fan
- Cisco Temperature Sensor
- Cisco Memory Pool
Groups
- Cisco Devices
- Cisco Ethernet Interfaces
- Cisco Serial Interfaces
Monitors
- Cisco Device Default GW Changed Monitor:
Monitor that detects and alerts on changes to the device’s default gateway, as defined in the IP route table. - Cisco Device CPU Utilization Monitor:
Monitor that generates an alert on high CPU utilization. - Cisco Interface Status Monitor:
Monitor that generates an alert when the OperStatus for the interface is not up. Alert is only generated when AdminStatus is up. - Cisco Serial Interface Flapping Monitor:
Monitor that generates an alert when the ifResets counter for the interface increases by more than 2 in a polling cycle. - Cisco Interface Collisions Monitor:
Monitor that generates an alert when the collisions reported on an interface increase by more than 25 in a polling cycle. - Cisco Interface Utilization Monitor:
Monitor that generates an alert when the percentage of inbound or outbound utilization is above 85. - Cisco Power Supply Status:
Monitor that generates an alert when the power supply status is not ok. - Cisco Fan Status Monitor:
Monitor that generates an alert when the fan status is not ok. - Cisco Temperature Sensor Status Monitor:
Monitor that generates an alert when the temperature sensor status is not ok. - Cisco Memory Pool Utilization Monitor:
Monitor that generates an alert when the utilization of the memory pool is greater than 90 percent.
Rules
- Collect Cisco CPU Utilization 5min:
Rule that collects the average CPU load over the past five minutes for performance reporting - Alert on Cisco Device Rebooted:
Rule that generates an alert when the SysUptime is less than 10 minutes, indicating a reboot - Collect Interface In Utilization:
Rule that collects the percentage of inbound interface utilization for performance reporting, calculated from the change in ifInOctets in the past polling cycle and divided by the interface speed. - Collect Interface Out Utilization:
Rule that collects the percentage of outbound interface utilization for performance reporting, calculated from the change in ifOutOctets in the past polling cycle and divided by the interface speed. - Collect Cisco Interface Collisions:
Rule that collects the number of collisions recorded for the interface in the last polling cycle for performance reporting. - Collect Cisco Memory Pool Utilization:
Rule that collects the percentage of memory pool utilization determined from the free and available bytes counters, for performance reporting - Collect Cisco Temperature Sensor Data:
Rule that collects the reported temperature value of a temperature sensor for performance reporting.
Views
- Cisco Device Alerts: Alerts view
- Cisco Device Performance: Performance view
- Cisco Devices: State view
- Cisco Fans: State view
- Cisco Interfaces: State view
- Cisco Memory Pools: State view
- Cisco Power Supplies: State view
- Cisco Temperature Sensors: State view
About Kristopher Bash
Operating-Quadrant 
Hi Kris!
Thanks man! Deep respect. Hope you don’t mind I have posted an article on my blog reffering to this posting of yours: http://thoughtsonopsmgr.blogspot.com/2009/09/free-cisco-mp.html
Keep up the good work.
Best regards,
Marnix
Of course, I don’t mind. Thanks!
Thanks a lot Kris for this great management pack. I hope you come out with the SP1 MP soon..
Hopefully by the end of the week. The monitors and classes will be identical, I just need to replace the discoveries with WMI-based script discoveries. Thanks!
Impressive!
Thanks!
Great work! Will try it today
Let me know how it works out. I’m anxious to learn if my testing was sufficient.
Hi Kris.
Implemented it at a customerssite yesterday and as a test only added one (BIG) Cisco device in OpsMgr. Really works very good. Awesome. The SNMP device is picked up fast by the Cisco SNMP MP and within some minutes all components were neatly detected and monitored as well. Also the diagram view looks very neat and professional. Thanks again for sharing this kind of high quality MP.
First off, thanks for providing this to the community!
I do have a few observations. It can take a little bit for the Interface Descriptions to get populated beyond just the Index # and IP Address, this can cause duplicate Performance Counters, one with an instance name, and one without.
Would it be possible to only discover certain interfaces instead of all of them? Say for device1 I only care about Interface 1,2,3 & 50, would it be possible to exclude all the others from discovery?
Thanks Again!
Thanks, your observations are helpful. I’ll work on the performance counters issue, as that could be annoying. What I’ll probably do is just filter the Performance Counters from writing to the DB if the Interface properties haven’t fully been discovered. As for filtering interface discovery, that’s a great idea. I can add a RegEx filter on the discovery, so that a Regular Expression can be passed as an override. Something like ^(1|2|3|50)$ to discover just those numbered interfaces.
I’ll probably post a new revision soon including these items, once I make the modifications and test. Thanks again
I have modifed the MP in the latest version to not collect data during the time the interface has been discovered, but all properties have not been discovered. You can adjust the intervals if need be, as by default there may be a 10 minute gap between the interface discovery (every 12 hours) and the property discovery. The property discovery is just an SNMP poll, so there’s no real harm in having it running more frequently if 10 minutes, twice a day, without data collection is too long.
I’ve also added the ability to filter interface discovery for a device with an override. Check out the latest post, and thanks for your comments.
Super cool, so glad to see people helping out. Any plans to build for other devices:-) HP Procuve is something I would like.
Thanks again.
Björn
It should be pretty easy to port the methods in this MP to other devices. My plans were to develop a Net-SNMP and CheckPoint SPlat MP (hardware only, the Net-SNMP MP would cover O/S stuff) next, maybe followed by a Sun hardware MP.
I could put together an HP Procurve version, but I’m limited in testing to what I can simulate in the free SNMP simulator I am using. I don’t have acces to any Procurve devices. What device models/series are you interested in monitoring? What items do you want to monitor?
Kris,
Whatever you have time to add is good but I really like what you have added for the Cisco MP. If you use the Jalasoft simulator you have a HP Procurve and I can help you test on other models, just send me info and I test.
Keep up the good job.
Björn
Kristopher,
Any progress in porting this to some other devices:-) I have customers that would really like to get some monitoing on the HP Procurve. Let me know if I can help you in any way to test (I can get some hardware for testing). Send ma an mail – if it would be interesting.
Yes, I have made some substantial progress in this direction. I’ll email you and certainly enlist you in testing if you are interested.
Really nice MP!
As Richard posted the selective interfaces would be great. I added a cisco 2841router and adds all the BRI sub interfaces like BRI0/3/1-Signaling that are more down then up. Little annoying, so yes it would be nice to include or exclude interfaces on a per device bases.
Great stuff.
Have a look at the latest post, this has been added.
Very impressive – Really good job!
I showed it to our network guys and they like it as well. Now there is one question: Is it possible / easy to add the interface descriptions they entered for all the interfaces? This would make it much easier to understand what we are looking at.
Again: Great job!
Glad to hear your initial impressions are positive. The Interface descriptions should be discovered. To display the descriptions in the Interfaces view, you can do one of two things. The easy way is to Personalize the view: right click the Cisco Interfaces view, choose Personalize View, and check the Description column. These view settings are saved on a per-user basis. To include the column in this view by default for all users, the MP needs to be edited (I’ll correct this in the next release of the MP). If you want to edit the default display of this column, you can open the XML of the MP, find the view named CiscoSNMP.View.Interfaces under the Presentation category. Under this view, drill into /Presentation and find the ColumnInfo element that has a Name and ID value of “Description.” For this ColumnInfo element, change the “Visible” attribute to True.
Then you would just re-import the MP. Hope this helps
Wow – you are fast. 🙂
Sorry, I am not a networker so I expressed it the wrong way. We are looking for the ifalias of each interface (port). I found it defined in the Cisco IF-MIB – see: http://tools.cisco.com/Support/SNMP/do/BrowseOID.do?local=en&translate=Translate&objectInput=ifAlias
That should not be a problem. There are some updates I’m planning on making in the next few days, I’ll include that one as well.
Sounds good.
May I add “hostName” to the wish list?
Sure, but can you clarify? Do you mean the hostName value in the old-cisco-mib http://tools.cisco.com/Support/SNMP/do/BrowseMIB.do?local=en&mibName=OLD-CISCO-SYSTEM-MIB? Can you confirm the OID?
Yes, that’s exactly what I mean. The OID is 1.3.6.1.4.1.9.2.1.3 according to Cisco.
very nice management pack!
great work!
Thank you!
I have a problem with Temperature sensors on Cisco C2960G and C2950. OID specified in managment pack is not correct for those two switch models.
Correct OIDs:
2960:
.1.3.6.1.4.1.9.5.1.2.13.0
2950:
.1.3.6.1.4.1.9.9.10.1.1.1.0
I tried to fix it, but no luck.
Also on C2960 switches interfaces have higher OID so port 1 is on OID: 10101 not 1 at the end. So to detect these interfaces simple fix has to be made in “Dicsover Cisco Interfaces”:
(.|..|…|..1..)
Thanks for these comments. I am working on an updated version, so I’ll look into the temperature monitoring. The RegEx filter can also be set to (.+) to match all interfaces regardless of the length of their index. I will probably use this as the default in the next version.
The OID you specified for the 2960 is in the Cisco-Stack MIB. It is an alarm state value along with alarm values for fans and minor and major chassis alarms. I can add monitors to check those OIDs, but I don’t see the benefit to actually discovering objects for them because they are device-level values. Monitoring the values for an alarm state should suffice.
Can you check the OID that you specified for the 2950? That is in the Cisco-Flash Mib and doesn’t appear to be valid. http://tools.cisco.com/Support/SNMP/do/BrowseMIB.do?local=en&step=2&mibName=CISCO-FLASH-MIB
There is no temperature sensor in C2950, so OID I specify in previous comment is incorrect.
Hey Kristopher,
Do you think you will have the SP1 version coming out anytime soon?
Thanks
Sameer
I’ve been working on it this week and hope to have it done by this weekend.
Hey Kristopher,
I’ve have a little problem with the Stack switch Fan alarm. It’s report a warning in SCOM and the oid value is 2. If you lookup that value its means ok. It’s a cisco 3750E
This is fixed in version 1.0.2.7, available on the downloads page now. This fix was the only change in that version, so if you have a number of overrides that you’d like to preserve, I can tell you where to correct the condition detection in the XML if you wanted.
I would like to know where to fix xml.
Find the UnitMonitorType with the ID: CiscoSNMP.MonitorType.StackFanAlarm.
This is under: ManagementPack -> TypeDefinitions -> MonitorTypes. Under the UnitMonitorType, there are two ConditionDetection Modules with ID’s CDFanAlarmOn and CDFanAlarmOff
The path is UnitMonitorType – > MonitorImplementation -> MemberModules
The integer value in the ValueExpression/Value element for each of these Condition Detection modules needs to be modified to: 2.
See this screenshot:
Hi, Thanks for the MP, works great so far. 1 quick questions. When I added some divices, it started to alert me on ports that I don’t have stuff connected to. Is there a way to have this available, but not alert on disconnected ports? I don’t want to disable the Cisco Interface completely because I want to stil be able to see my ports and stuff I have connected to them.
Thanks
Well, the MP alerts on interfaces when their admin status is up and their operational status is down. There are three ways to avoid alerting on interfaces that are not used
1) Create a group for unmonitored interfaces, add the interfaces you don’t want alerts on, and override the alerts to be disabled for this group
2) shutdown the interfaces on the Cisco devices (with the shut command), which changes their admin status to down
3) Override the interface discovery to only include the interfaces you want to monitor.
It sounds to me like the first option might be closest to what you are looking for. I hope this helps
Wow,
thank you very much for this. You have prevented me putting a lot of work in authoring an own management pack on our site. The management packs work fine and does a great job!! It`s a pitty that I had to disable the rule for checking the interface status monitor because in default our ports are set to up by admin. So I prevent 100`s of critical errors:-)
Thanks once again..
kind regards
I’m currently hard at work on version 1.0.3 of this MP which will have a number of improvements. There are a couple of ways to approach the interface situation. 1) You can use the RegEx discovery filter on the interface discovery to restrict which interfaces are discoverd. 2) You can create a group for interfaces to monitor and disable the monitors by default but enable it with overrides for the group. 3) you can do the reverse and create a group for disabled interfaces and override the monitors to disable them for this group.
I will be distributing version 1.0.3 with an overrides MP that has these groups and overrides in place to make it easier.
Pingback: Cisco Management Pack für OpsMgr | Noebauer Johannes // Blog
Hi,
Great MP, I was interested in being able to create a group to only monitor certain interfaces, you’ve used groups and overrides to do this. I would like to build a query to dynamically assign interfaces based in the ifalias (i.e. if the ifalias contains any data then the interface should be monitored) when you try to pick the desired class, all classes are shown as ‘interface’
Would it be possible to show the correct class ?
Thanks
Paul
Hi, all. I was very happy to have found this post and was almost starting modifying the MP to make one for HP procurve switches.
But I’ve seen that you are already working on it, if i can help out i would be glad to. We have HP procurve 2600’s (2610) and 5400’s (5406zl) and some 2900’s.
Thanks for the comment. The Management Packs I’m working on are currently in somewhat of a beta-testing stage. If you are interested in testing them out, email me at: opquad@ live.com.
Thanks again
Well done on the MP. I am just trying it out now and very impressed. Reading through the comments regarding interfaces alerting as ‘down’ when nothing is connected all you need to do is add the comment ‘no snmp trap link-status to the interfaces you do not want to alert such as workstations and or servers. This will prevent snmp traps being sent from the switch to the management station. I then deleted the devices and discovered them again and all seems to be ok and healthy.
In addition to my last comment I take it that everyone knows to add the server host to the snmp config too in order for traps to be sent to the correct/ relevent destinations.
We love this management pack. We where wanting to make a couple changes though, in the diagram view, how can we change the interface names to show the descrition instead. We dont name our interfaces, we simply look for the description which is FastEthernet0/12, etc.
Also, in the Cisco Device view, how can we change to show the name field by default instead of the hostname? The management pack looks to work great for the MDS fibrechannel switches, with the excpetion that they dont have a hostname, just the name.
Thanks for a great management pack.
First off – really neat MP
Got an issue – I’ve seen scenarios where all the interface performance monitors for a majority of my devices disappear, sometimes for days at a time, then re-appear. One of my co-workers is piloting OpenNMS and he’s not seeing anything out of the ordinary. I’m guessing it’s something to do with discovery – any ideas?
The only issue I have is my async interfaces which are used for remote admin purposes come up as critical errors since they are not shutdown but are down/down…
This is a quality piece of work. Really pro.
Well done.
Pingback: SCOM: Advanced SNMP Monitoring Part III: The Completed Cisco Management Pack « Operating-Quadrant - Rod Trent at myITforum.com
hi Kristopher,
Nice work!. But how do you address the octet counter overflow issue.? In most of the routers/switches the OS is 32Bits so if you have a polling time of lets say 5 minutes the change is big the counter value will reset from 2^32 to 0 . This reset has to be addressed in the utilization calculation, otherwise you can have a in correct utilization when the reset has occurred between T0 and T1.
Michel Kamp
This is corrected in the xSNMP Management Packs (which I have made a couple of posts about this week). The xSNMP MP’s address this issue by doing the following:
1) Discovering each interface’s support for 64 bit counters
2) For interfaces that support 64 bit counters, an override is automatically used to override the OID’s for IF octets to use the IF-MIB 64 bit counters in the calculations.
This way 32 bit counters can still be used for low-speed interfaces, but if the agent supports 64 bit counters, they will automatically be used. For agents that don’t support 64 bit counters on high speed interfaces, I also configured the calculation script in these MP’s to discard the calculation if the delta is negative, which will at least prevent negative utilization values from being logged when a counter does rollover.
Hi Kristopher,
I just installed your MP on our test network and I am a newbie to cisco. I have about 15 adminstatus is up alerts. Could you please explain this alert. I few of the prots are empty but the rest are in use and seem to be working fine.
Thanks,
Tim
A question from our Network team…
“Would it be possible to get the Cisco interface monitor to pull in/report IF-MIB::ifDescr (1.3.6.1.2.1.2.2.1.2.x)? It would make it far easier to understand what was going on if we could see the interface name. User-friendly reporting of the administrative/operational status in the health explorer would be nice too – having to decode both the OIDs and their values is a bit nerdy.”
Any ideas what he is on about? 🙂
Allen, if you are on SCOM R2, I would highly recommend looking to replace this Cisco MP with the xSNMP Management Packs, also available from my blog site.
excellent travail, merci
Merci de m’indiquer le lien ou je pourrais télécharger le MP xSNMP qui me permettra de monitorer les équipements Cisco
Merci pour votre aide
Hi
Great work with this MP. The only thing which annoys me is that i cant get monitored interfaces of CISCO switches.
Is there any way to enforce monitoring of the interfaces?
Thanks,
We’ve had problems with running xSNMP in our environment, where (in a tight nutshell) it has caused Gateways to get corrupted Event Logs.
Is the Cisco Management Pack still viable, or are there any other options for doing network device interface monitoring? We don’t really have the resources right now to troubleshoot xSNMP further, and have uninstalled that MP.
-Randall
First off I would like to say fantastic blog!
I had a quick question which I’d like to ask if you don’t mind.
I was curious to find out how you center yourself and clear your thoughts prior to writing.
I’ve had a tough time clearing my mind in getting my ideas out. I truly do take pleasure in writing but it just seems like the first 10 to 15 minutes are generally wasted simply just trying to figure out how to begin. Any ideas or hints? Appreciate it!