Integrating Elasticsearch and Logstash with NetApp Harvest and Grafana : Part 3

Forwarding NetApp's Syslog to Logstash

Hopefully your setup of ES and LS (Part 2 of this series) was a success.  If everything is up and listening then let's start sending some logs to your server.

Configuring CDOT v. 8.2, 8.3

Pushing your NetApp logs to your ES/LS server is real easy.  Lets start with CDOT 8.3 and 8.2 below.

You first need to create a destination name for your logs and tell OnTap where to send your them:

 event dest create -name netapplog -syslog yourLSservername  

You then need to route which messages you want to send to your new destination.  Be careful!  Your first instinct will to send everything!

 event route add-destinations -destinations netapplog -message *  

If your cluster(s) has a good number of nodes or the workload is pretty heavy then you will most likely drown in logs.  Most of which you don't need.  If you don't mind or you have a fairly small cluster that you are just testing on then you should be okay.  This will, at the least, give you an idea of what to expect and you can tailor what you send based on what you see or prefer.
Remember that CDOT has over 7000 different messages that can be triggered.  Maybe you or your customer are only interested in specific events. I don't have a list of all of them but you could run this command to view them.

 event route show -messagename *  

Say you are only interested in vserver messages:

 event route add-destinations -dest netapplog -message vserver*  

Integrating Elasticsearch and Logstash with NetApp Harvest and Grafana : Part 2

Setting Up Your SYSLOG Server

Install Elasticsearch and Logstash

Since this blog series is about how to integrate ES and LS with NetApp and the NAbox,  I won't take the time to explain the installation of each.  Installation of both is fairly easy and there are many sites and blogs that explain it in detail.  I'll just add a few links and notes below that I found very helpful.

Elasticsearch Install Link                    Logstash Install Link

Some basic info about my instance:
OS:  Ubuntu 14.04.3 LTS
ES version 2.4.2
LS version  2.4.1
Installing Elasticsearch as a single node on the server
Elasticsearch and Logstash require java.  Refer to the install links above to verify supported versions.
Clustered DataOntap (CDOT) 8.2.4 and 8.3.1

Configure Elasticsearch

Integrating Elasticsearch and Logstash with NetApp Harvest and Grafana : Part 1

Digging even deeper...

Background

I've been using a great tool called NetApp Harvest for a few years now.  If you are familiar with Harvest and how it integrates with Grafana and Graphite then you know what this tool can provide you and/or your customers.  A colleague of mine at NetApp , Chris Madden (@dutchiechris on twitter) is the author and maintainer of NetApp Harvest and you can grab all the info you need on it from his blog called "it all begins with data".  One of the great things about Harvest is that it comes with  a ton of dashboards ready to install on Grafana!  Another colleague of mine, Yann Bizeul (@ybontap on twitter) built an "all-in-one" OVA template called the NAbox (formerly called ADVA).  The NAbox makes deploying Harvest/Grafana/Graphite very easy.  In minutes you can get it up and running and see beautiful metrics pouring out from your NetApp systems.  All the information you need to deploy this wonderful tool is located here https://nabox.tynsoe.org/.

NetApp Dashboard: Cluster Group