Using the Instaclustr Monitoring API with Data Dog

Instaclustr’s monitoring API is designed to allow you to integrate monitoring information from your Instaclustr managed cluster with the monitoring tool used for entire application. DataDog (datadoghq.com) is a popular platform for monitoring a range of applications. This help article walks you through how to use the Instaclustr Monitoring API with Data Dog.

At a high-level, the approach we will take in this article is to install a script on a server you manage that has the DataDog agent installed. This script calls the Instaclustr Monitoring API on regular intervals and passes the information returned to the DataDog agent which reports it to the central DataDog system.

One of our awesome customers has also come up with and uses an alternative approach using AWS lambdas. You can find details here: https://github.com/manheim/InstaCluster-to-Datadog-Lambda

Prepare Your Environment

Follow these steps to set up your environment:

  1. Set up a cluster with Instaclustr (see https://support.instaclustr.com/hc/en-us/articles/203738340-Creating-a-Cluster)
  2. Set up a DataDog account (datadoghq.com)
  3. Installed the DataDog agent on the machine you will use to run the integration script. (Install instruction are available in the DataDog console.)
  4. Installed Python on that machine (https://www.python.org/downloads/)
  5. Installed the pip Python package manager on the machine (https://pip.pypa.io/en/stable/installing/).
  6. Install the DataDog DogStatsD API package (pip install datadog). 

Set Up The Script

We have created a sample script that calls the Instaclustr API and forwards the data to DataDog. The script is available on GitHub here: https://github.com/instaclustr/ICAPI-DataDog

The script (ic2datadog.py) itself is fairly straightfoward. It retrieves a specified list of metrics for all nodes in the cluster and It requires a configuration file (called configuration.json) in the format shown below:

{
  "cluster_id":"[instaclustr cluster id]",
  "metrics_list":"n::cpuutilization,n::cassandraReads,n::cassandraWrites,n::nodeStatus",
  "dd_options":
  {
    "api_key":"[datadog API key]",
    "app_key":"[datadog app key]"
  },
  "ic_options":
  {
    "user_name":"[instaclustr user name]",
    "api_key":"[instaclustr password]"
  }
} 

The settings required here are:

  • cluster_id: the Instaclustr cluster ID, available from the cluster details page on the Instaclustr console
  • metrics_list: a comma separated list of metrics to retrieve and pass to DataDog. For a full list of available metrics, see the Instaclustr monitoring API documentation.
  • dd_options: your DataDog api key and app key, available from Integrations/APIs in the DataDog console (you may need to create a new app key).
  • ic_options: your Instaclustr user name and API key, available from https://www.instaclustr.com/dashboard/profile/apikey

Run The Script and View The Results

Running the script is a simple matter of ‘python ic2datadog.py’. The script will then run until interrupted.After a minute or so of running, the metrics will be visible in DataDog.To view the guage metrics (eg CPU), log in to the DataDog console and:

  1. Go to Metrics/Explorer
  2. In the Graph text box, start typing ‘instaclustr’. You should see a list of available metrics appear in the format “Instaclustr.[Node IP].[metric name]. Choose the metrics you want and DataDog will draw you a graph.

To view the node status information:

  1. Go to Monitors/Check Summary
  2. You should see the Instaclustr node status checks in the list (filter for “Instaclustr” if necessary). 

The Instaclustr metrics are now available to use wherever else you would use them in Data Dog (dashboards, monitors, etc).

Last updated:
If you have questions regarding this article, feel free to add it to the comments below.

0 Comments

Please sign in to leave a comment.