Build

Build Incident Management In Real time Apps with PagerDuty

8 min read • Michael Carroll on Dec 22, 2016

PagerDuty is a cloud-based platform that provides incident management and response across a huge range of third-party operational services (such as Pingdom, Splunk, StatusCake, New Relic, and many, many more). With PagerDuty, developers have a single pane of glass for managing events across the full range of platforms. It’s also easy to create a custom integration that integrates your custom system with PagerDuty operational response.

We’re excited to announce that developers can now harness the power of PagerDuty combined with their real-time streams with our new PagerDuty block. For example, if you have a field of temperature sensors and the reading reaches a specific threshold, you can alert your support team to carry out actions with PagerDuty’s wide variety of incident management features. Or if your chat app is powered by PubNub Pub/Sub Messaging, and a certain number of errors are returned, PagerDuty is activated and actions can be taken to resolve the issue.

As the number of cloud-enabled applications grows exponentially, the efficient management of technical events (like performance, scaling and outages) as well as business events (such as business process blocked) becomes more and more critical. With PagerDuty, it is very easy to manage incident response and escalation across a collection of services and team members.

Incident Management Tutorial Overview

In this article, we dive into a simple example of how to trigger PagerDuty operational events from an HTML5 AngularJS application with a modest 27-line PubNub JavaScript BLOCK and 67 lines of HTML and JavaScript.

PagerDuty in Action

 

As we prepare to explore our sample AngularJS web application with DevOps integration, let’s check out the underlying PagerDuty API.

PagerDuty API

DevOps can be messy business, and PagerDuty brings considerable order to that chaos. When a typical cloud product uses tons of service dependencies for both monitoring and primary services, PagerDuty is the hub that collects and organizes all of the necessary information and functionality for operational response in one place. The PagerDuty API features a number of endpoints for escalation policies, incidents, log entries, maintenance windows, schedules and notifications.

Getting Started with PagerDuty API

The next thing you’ll need to get started with the DevOps APIs is a PagerDuty account to take advantage of the PagerDuty APIs.

  • Step 1: go to the PagerDuty signup form to create an account.
  • Step 2: go to the PagerDuty dashboard, create a new “service”, and a new “integration” under that service (make note of the integration key for the future).
  • Step 3: configure user escalation in your account notification rules if you like (including email, SMS, mobile push notification and phone call escalation).

Setting up the BLOCK

With PubNub BLOCKS, it’s really easy to create code to run in the network. Here’s how to make it happen:

Create BLOCK
  • Step 2: create a new block.
Create Event Handler
  • Step 3: paste in the block code from the next section and update the credentials with the integration key from the previous step above.
Paste PagerDuty Code
  • Step 4: Start the block, and test it using the “publish message” button and payload on the left-hand side of the screen.

That’s all it takes to create your serverless code running in the cloud!

Diving into the Code – the BLOCK

You’ll want to grab the 27 lines of BLOCK JavaScript and save them to a file, say, pubnub_pagerduty_block.js. It’s available as a Gist on GitHub for your convenience.

First up, we declare our dependency on xhr (for HTTP requests) and create a function to handle incoming messages.

export default (request) => {
    const xhr = require('xhr');

Next, we set up variables for accessing the service (the integration key and API URL from previous steps).

    const integrationKey = 'YOUR_INTEGRATION_KEY';
    const apiUrl = 'https://events.pagerduty.com/generic/2010-04-15/create_event.json';

Next, we extract the description from the message body.

    const description = request.message.description;

Next, we set up the HTTP options for the incident API request. We use a POST request to submit the data. We use the integration key to identify and route the incident creation request to the proper service in PagerDuty. We create a random “incident key” to correlate our notion of the event with the internal PagerDuty event ID.

    const httpOptions = {
        method: 'POST',
        body: JSON.stringify({
            service_key: integrationKey,
            event_type: 'trigger',
            incident_key: "incident_" + parseInt(Math.random() * 10000),
            description: description
        })
    };

Finally, we call the incident trigger endpoint with the given data, decorate the message with a result value containing the incident data, and catch any errors and log to the BLOCKS console. Pretty easy!

      return xhr.fetch(apiUrl, httpOptions)
      .then((response) => {
          console.log(response);
          request.message.result = response;
          return request.ok();
      })
      .catch((e) => {
          console.error(e);
          return request.ok();
      });
  }

All in all, it doesn’t take a lot of code to add DevOps incident management to our application. We like that!

OK, let’s move on to the UI!

Diving into the Code – the User Interface

You’ll want to grab these 67 lines of HTML & JavaScript and save them to a file, say, pubnub_pagerduty_ui.html.

The first thing you should do after saving the code is to replace two values in the JavaScript:

  • YOUR_PUB_KEY: with the PubNub publish key mentioned above.
  • YOUR_SUB_KEY: with the PubNub subscribe key mentioned above.

If you don’t, the UI will not be able to communicate with anything and probably clutter your console log with entirely too many errors.

For your convenience, this code is also available as a Gist on GitHub, and a Codepen as well. Enjoy!

Dependencies

First up, we have the JavaScript code & CSS dependencies of our application.

<!doctype html>
<html>
<head>
  <script src="https://code.jquery.com/jquery-1.10.1.min.js"></script>
  <script src="https://cdn.pubnub.com/pubnub-3.15.1.min.js"></script>
  <script src="https://ajax.googleapis.com/ajax/libs/angularjs/1.5.6/angular.min.js"></script>
  <script src="https://cdn.pubnub.com/sdk/pubnub-angular/pubnub-angular-3.2.1.min.js"></script>
  <script src="https://cdnjs.cloudflare.com/ajax/libs/underscore.js/1.8.3/underscore-min.js"></script>
  <link rel="stylesheet" href="//netdna.bootstrapcdn.com/bootstrap/3.0.2/css/bootstrap.min.css" />
  <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.6.3/css/font-awesome.min.css" />
</head>
<body>

For folks who have done front-end implementation with AngularJS before, these should be the usual suspects:

  • JQuery : gives us the ability to use easy JQuery selectors.
  • PubNub JavaScript client: to connect to our data stream integration channel.
  • AngularJS: were you expecting a niftier front-end framework? Impossible!
  • PubNub Angular JavaScript client: provides PubNub services in AngularJS quite nicely indeed.
  • Underscore.js: we could avoid using Underscore.JS, but then our code would be less awesome.

In addition, we bring in 2 CSS features:

  • Bootstrap: in this app, we use it just for vanilla UI presentation.
  • Font-Awesome: we love Font Awesome because it lets us use truetype font characters instead of image-based icons. Pretty sweet!

Overall, we were pretty pleased that we could build a nifty UI with so few dependencies. And with that… on to the UI!

The User Interface

Here’s what we intend the UI to look like:

PagerDuty UI

The UI is pretty straightforward – everything is inside a div tag that is managed by a single controller that we’ll set up in the AngularJS code.

<div class="container" ng-app="PubNubAngularApp" ng-controller="MyDevOpsCtrl">
<pre>
NOTE: make sure to update the PubNub keys below with your keys,
and ensure that the devops BLOCK is configured properly!
</pre>
  
<h3>MyApp DevOps Controls</h3>

We provide a simple button for the “PANIC” trigger by a user which performs the AngularJS controller’s publish() action.

<input type="submit" ng-click="publish()" value="PANIC!" />

Our UI consists of a simple list of data messages. We iterate over the messages in the controller scope using a trusty ng-repeat. Each message includes the “panic” description and integration key returned by the PagerDuty API.

<ul>
  <li ng-repeat="message in messages track by $index">
    Event Description: {{message.description}}
    <br />
    PagerDuty Incident Key: {{message.result.incident_key}}
  </li>
</ul>

And that’s it – a functioning real-time UI in just a handful of code (thanks, AngularJS)!

The AngularJS Code

Right on! Now we’re ready to dive into the AngularJS code. It’s not a ton of JavaScript, so this should hopefully be pretty straightforward.

The first lines we encounter set up our application (with a necessary dependency on the PubNub AngularJS service) and a single controller (which we dub MyDevOpsCtrl). Both of these values correspond to the ng-app and ng-controller attributes from the preceding UI code.

<script>
angular.module('PubNubAngularApp', ["pubnub.angular.service"])
.controller('MyDevOpsCtrl', function($rootScope, $scope, Pubnub) {

Next up, we initialize a bunch of values. First is an array of message objects which starts out empty. After that, we set up the channel as the channel name where we will send and receive real-time structured data messages.

NOTE: make sure this matches the channel specified by your block configuration!

$scope.messages  = [];
$scope.channel   = 'pagerduty-event';

We initialize the Pubnub object with our PubNub publish and subscribe keys mentioned above, and set a scope variable to make sure the initialization only occurs once.

NOTE: this uses the v3 API syntax.

  if (!$rootScope.initialized) {
    Pubnub.init({
      publish_key: 'YOUR_PUB_KEY',
      subscribe_key: 'YOUR_SUB_KEY',
      ssl:true
    });
    $rootScope.initialized = true;
  }

The next thing we’ll need is a real-time message callback called msgCallback; it takes care of all the real-time messages we need to handle from PubNub.

In our case, we have only one scenario – an incoming message containing the “panic” event. We unshift the message object onto the scope array; that unshift() operation should be in a $scope.$apply() call so that AngularJS gets the idea that a change came in asynchronously.

  var msgCallback = function(payload) {
    $scope.$apply(function() {
      $scope.messages.unshift(payload);
    });
  };

The publish() function takes the contents of the select inputs and publishes them as a structured data object to the PubNub channel that the BLOCK will understand.

  $scope.publish = function() {
    Pubnub.publish({
      channel: $scope.channel,
      message: {description:"user hit PANIC button on channel '" + $scope.channel + "'!"}
    });
  };

In the main body of the controller, we subscribe() to the message channel (using the JavaScript v3 API syntax) and bind the events to the callback function we just created.

  Pubnub.subscribe({ channel: [$scope.channel], message: msgCallback });

We mustn’t forget close out the HTML tags accordingly.

});
</script>
</body>
</html>

Now, we have a direct integration between our real-time message channel and the PagerDuty incident management API. If you’re building an application on another platform, you would follow exactly the same PubNub message pattern, with the benefit that no data in the BLOCK would need to change!

Not too shabby for a little less than seventy lines of HTML & JavaScript!

Conclusion

Thank you so much for joining us in the PagerDuty DevOps services article of our web service integration series! Hopefully it’s been a useful experience learning about operationally-aware technologies. In future articles, we’ll dive deeper into additional web service APIs and use cases for other real-time web applications.

Stay tuned, and please reach out anytime if you feel especially inspired or need any help!

0