Streaming Data and Metrics Into New Relic Insights

New Relic provides performance monitoring and insights for applications to track, display, and disseminate metrics quickly, providing rapid insight and rapid response capabilities. They also offer a number of services for creating, displaying and alerting on metrics.

Our New Relic BLOCK allows you to insert data being streamed over the PubNub network into the New Relic Insights service. This might be customer data on a retail app, allowing you to track activity and purchases and visualize the customer journey, all in real time. Or it could be IoT sensor readings, and you could trigger a certain alert or notification if it crosses a certain threshold.

New Relic BLOCK Tutorial Overview

In this tutorial, we’ll dive into a simple example of how to track and display custom metrics from a real-time Angular2 web application using the New Relic BLOCK. As anyone running a large-scale application will attest, tracking technical and business metrics is vital to ensuring continuity of service, business health, and recovering from issues quickly.

As we prepare to explore our sample Angular2 web application with custom monitoring, let’s check out the underlying New Relic Insights API.

New Relic Insights API

Large-scale monitoring and metrics analysis is a notoriously difficult area, and it requires substantial effort and engineering resources to maintain high availability and high efficiency in a metrics collection system. New Relic provides a solution to these issues with a friendly UI and easy-to-use API that “just works.” Through the

New Relic provides a solution to these issues with a friendly UI and easy-to-use API that “just works.” Through the API and user interface, it is possible to track new metrics, create charts from simple and compound metrics, organize charts into dashboards, and query dynamic data using New Relic Query Language.

Getting Started with New Relic

To get started with New Relic services you’ll need a New Relic developer account to take advantage of the monitoring APIs.

Go to the New Relic signup form.
Go to the APM page and configure an application agent.
Go to the Account Subscription page under “Account Settings” and start a trial of the Insights API.
Go to the insights page and make note of your API Key and Account ID.

Overall, it’s a pretty quick process. And free to get started, which is a bonus!

Setting up the BLOCK

With PubNub BLOCKS, it’s really easy to create code to run in the network. Here’s how to make it happen:

Go to the application instance on the PubNub admin dashboard.

Create a new BLOCK.

Paste in the BLOCK code from the next section and update the Account ID and Insights Key with the values from the previous steps above.

Start the BLOCK, and test it using the “publish message” button and payload on the left-hand side of the screen.

That’s all it takes to create your serverless code running in the cloud!

Diving into the Code – the BLOCK

You’ll want to grab the 31 lines of BLOCK JavaScript and save them to a file called pubnub_newrelic_block.js. It’s available as a Gist on GitHub for your convenience.

First up, we declare our dependency on xhr (for HTTP requests) and create a function to handle incoming messages.

export default request => {
    let xhr = require('xhr');
    let query = require('codec/query_string');

Next, we set up variables for accessing the service (the API url and credentials).

    var accountId = 'YOUR_ACCOUNT_ID';
    var insightsKey = 'YOUR_INSIGHTS_KEY';
    var url = 'https://insights-collector.newrelic.com/v1/accounts/'
        + accountId + '/events';

Next, we set up the HTTP params for the Insights API URL. We use a POST request for the URL. We use the JSON content of the message as the source of most important parameters, and set the authentication headers.

    var params = {
        method: 'POST',
        headers: {
            'X-Insert-Key': insightsKey,
            'Content-Type': 'application/json'
        },
        body: request.message
    };

Finally, we POST the given data and decorate the message with a newrelic_result attribute containing the results of the API call. Pretty easy!

    return xhr.fetch(url, params)
        .then(function (resp) {
            request.message.newrelic_result = {
                headers: resp.headers,
                result: JSON.parse(resp.body)
            };
                
            return request.ok();
        }).catch((err) => {
            console.log('error happened for XHR.fetch', err);
            return request.ok();
        });
}

All in all, it doesn’t take a lot of code to add custom monitoring to our application. We like that!

OK, let’s move on to the UI!

Diving into the Code – the User Interface

You’ll want to grab these 87 lines of HTML & JavaScript and save them to a file called pubnub_newrelic_ui.html.

The first thing you should do after saving the code is to replace two values in the JavaScript:

YOUR_PUB_KEY: with the PubNub publish key mentioned above.
YOUR_SUB_KEY: with the PubNub subscribe key mentioned above.

If you don’t, the UI will not be able to communicate with anything and probably clutter your console log with entirely too many errors. For your convenience, this code is also available as a Gist on GitHub, and a Codepen as well. Enjoy!

Dependencies

First up, we have the JavaScript code & CSS dependencies of our application.

<!DOCTYPE html>
<html>
  <head>
    <title>Angular 2</title>
    <script src="https://unpkg.com/core-js@2.4.1/client/shim.min.js"></script>
    <script src="https://unpkg.com/zone.js@0.7.2/dist/zone.js"></script>
    <script src="https://unpkg.com/reflect-metadata@0.1.9/Reflect.js"></script>
    <script src="https://unpkg.com/rxjs@5.0.1/bundles/Rx.js"></script>
    <script src="https://unpkg.com/@angular/core/bundles/core.umd.js"></script>
    <script src="https://unpkg.com/@angular/common/bundles/common.umd.js"></script>
    <script src="https://unpkg.com/@angular/compiler/bundles/compiler.umd.js"></script>
    <script src="https://unpkg.com/@angular/platform-browser/bundles/platform-browser.umd.js"></script>
    <script src="https://unpkg.com/@angular/forms/bundles/forms.umd.js"></script>
    <script src="https://unpkg.com/@angular/platform-browser-dynamic/bundles/platform-browser-dynamic.umd.js"></script>
    <script src="https://unpkg.com/pubnub@4.3.3/dist/web/pubnub.js"></script>
    <script src="https://unpkg.com/pubnub-angular2@1.0.0-beta.8/dist/pubnub-angular2.js"></script>
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" />
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap-theme.min.css" />
  </head>

For folks who have done front-end implementation with Angular2 before, these should be the usual suspects:

CoreJS ES6 Shim, Zone.JS, Metadata Reflection, and RxJS : Dependencies of Angular2.
Angular2 : core, common, compiler, platform-browser, forms, and dynamic platform browser modules.
PubNub JavaScript client: to connect to our data stream integration channel.
PubNub Angular2 JavaScript client: provides PubNub services in Angular2 quite nicely indeed.

In addition, we bring in the CSS features:

Bootstrap: in this app, we use it just for vanilla UI presentation.

Overall, we were pretty pleased that we could build a nifty UI with so few dependencies. And with that… on to the UI!

The User Interface

Here’s what we intend the UI to look like:

The UI is pretty straightforward – everything is inside a main-component tag that is managed by a single component that we’ll set up in the Angular2 code.

<body>
  <main-component>
    Loading...
  </main-component>

Let’s skip forward and show that Angular2 component template. We provide a simple button to perform the publish() action and send a KERNEL

<div class="container">
    <pre>
    NOTE: make sure to update the PubNub keys below with your keys,
    and ensure that the BLOCK settings are configured properly!
    </pre>
    <h3>MyApp Insights Integration</h3>
    <hr/>
    <p>Click the button below to send an event</p>
    <button class="btn btn-primary btn-danger" (click)="publish()">Send KERNEL_PANIC!</button>
    <br/>
    <br/>
    <ul class="list-unstyled">
      <li *ngFor="let item of messages.slice().reverse()">
        <div>{{item.message.newrelic_result.headers.date}} : KERNEL_PANIC!</div>
      </li>
    </ul>
</div>

The component UI consists of a simple list of events (in our case, the KERNEL

And that’s it – a functioning real-time UI in just a handful of code (thanks, Angular2)!

The Angular2 Code

Right on! Now we’re ready to dive into the Angular2 code. It’s not a ton of JavaScript, so this should hopefully be pretty straightforward.

The first lines we encounter set up our application (with a necessary dependency on the PubNub AngularJS service) and a single component (which we dub main-component).

<script>
var app = window.app = {};
app.main_component = ng.core.Component({
    selector: 'main-component',
    template: `...see previous...`

The component has a constructor that takes care of initializing the PubNub service and configuring the channel name. NOTE: make sure the channel matches the channel specified by your BLOCK configuration and the BLOCK itself!

    }).Class({
        constructor: [PubNubAngular, function(pubnubService){
            var self = this;
            self.pubnubService = pubnubService;
            self.channelName = 'newrelic-channel';

Early on, we initialize the pubnubService with our credentials.

            pubnubService.init({
                publishKey:   'YOUR_PUB_KEY',
                subscribeKey: 'YOUR_SUB_KEY',
                uuid: PubNub.generateUUID(),
                ssl:true
            });

We subscribe to the relevant channel, create a dynamic attribute for the messages collection, and configure a blank event handler since the messages are presented unchanged from the incoming channel.

            pubnubService.subscribe({channels: [self.channelName], triggerEvents: true});
            self.messages = pubnubService.getMessage(this.channelName,function(msg){
                // no handler necessary, dynamic collection of msg objects
            });

We also create a publish() event handler that performs the action of publishing the new message to the PubNub channel.

        publish: function(){
            this.pubnubService.publish({channel: this.channelName, message:{eventType: "KERNEL_PANIC","server":this.pubnubService.getUUID(),"error": "I can't do that"}});
        }
    });

Now that we have a new component, we can create a main module for the Angular2 app that uses it. This is pretty standard boilerplate that configures dependencies on the Browser and Forms modules and the PubNubAngular service.

app.main_module = ng.core.NgModule({
    imports: [ng.platformBrowser.BrowserModule, ng.forms.FormsModule],
    declarations: [app.main_component],
    providers: [PubNubAngular],
    bootstrap: [app.main_component]
}).Class({
    constructor: function(){}
});

Finally, we bind the application bootstrap initialization to the browser DOM content loaded event.

document.addEventListener('DOMContentLoaded', function(){
    ng.platformBrowserDynamic.platformBrowserDynamic().bootstrapModule(app.main_module);
});

We mustn’t forget close out the HTML tags accordingly.

});
</script>
</body>
</html>

Not too shabby for about 87 lines of HTML & JavaScript!

Additional Features

There are a couple other endpoints worth mentioning in the New Relic Insights API. You can find detailed API documentation Clicking here.

Dashboards : ability to customize data display.
Applications : collections of linked dashboards.
Query : custom queries for Insights data.

All in all, we found it pretty easy to get started with custom monitoring using the API, and we look forward to using more of the deeper integration features!

Thank you so much for joining us in the custom monitoring article of our BLOCKS and web services series! Hopefully, it’s been a useful experience learning about DevOps-aware technologies.

Streaming Real-time Data and Metrics Into New Relic Insights