Build

The Right Way to Log JSON Messages to a Private Database

2 min read Stephen Blum on Feb 8, 2018
Try PubNub Today

Free up to 1MM monthly messages. No credit card required.

Subscribe to our newsletter

By submitting this form, you are agreeing to our Terms and Conditions and Privacy Policy.

With PubNub Functions, you can efficiently log all published message to your database.

At time of writing, PubNub does not index your JSON messages using FTS Indexing. Additionally you will want to log your JSON messages for future use cases. Your data is valuable. You may need full-text search indexing. Your database can do this. Common databases include MongoDB, MySQL, Oracle DB, PostgreSQL, SQLite and others. Also you may use an API provider like Algolia for full-text searching.

AI and ML allow you to create insight from your data using Tensorflow. You may want to run data analysis for the message content using EMR / Hadoop or other big data analysis software. This guide will help you save your JSON Messages into your data store for input vectorizing into an AI Model.

On StackOverflow, we received this question:

What is the right way to log every published message and save it to my server db.

There are two options which I can think of:

  1. Use PubNub function After Publish event and forward the message to a dedicated logger channel. the server will subscribe to the channel and save the arrived messages to db. Here rise another question: when I forward the message to another channel in the PubNub function will it also trigger the PubNub function?
  2. use PubNub XHR request/response function and call to a rest API on my server with the published message and save it to db

What is the best practice concerning performance and cost?

Log JSON Messages to Your Private Database

While many approaches exist, one is the best. Using PubNub Functions, you can asynchronously save JSON messages reliably to your database using an OnAfter Publish Event.

Note: your database needs to be accessible via a secured HTTPS interface.

Assuming you already have a stream of JSON messages being published to a PubNub data channel (chat messages, Bitcoin prices), follow these easy steps. You will create a real-time function that is triggered with every Publish event.

Here’s how you do it:

  1. Collect a list of channel(s). You may be interested in ALL channels *.
  2. In your Admin Dashboard find your app, then click Functions.
  3. Create a new Module for your specified API key set.
  4. Create a new OnAfter event handler on * channel.
  5. Use the following example code to save, asynchronously, your JSON messages to your database.
  6. Make sure to modify the URL/Parameters to fit your needs.

Asynchronous HTTPS Save Message Function

export default request => { 
    const xhr  = require('xhr');
    const post = { method : "POST", body : request.message };
    const url  = "https://my.company.com/save";
    // save message asynchronously
    return xhr.fetch( url, post ).then( serverResponse => {
        // Save Success! 
        return request.ok();
    }).catch( err => {
        // Save Failed! handle err
        return request.abort();
    });
}

More from PubNub

Comparing Game Engines: Unity vs Unreal vs the Rest
Real-Time Gaming BlogSep 21, 20225 min read

Comparing Game Engines: Unity vs Unreal vs the Rest

Comparing the major game engines: Unity vs Unreal Engine vs Corona SDK vs GameMaker Studio, including the benefits and cons of...

PubNub Staff

PubNub Staff

Python Socket Programming: Client, Server and Peer Libraries
BuildSep 21, 20226 min read

Python Socket Programming: Client, Server and Peer Libraries

Sockets (aka socket programming) enable programs to send and receive data, bi-directionally, at any given moment. This tutorial...

PubNub Staff

PubNub Staff

Node.js WebSocket Programming Examples
BuildSep 21, 20226 min read

Node.js WebSocket Programming Examples

An overview and examples of basic socket and WebSocket programming with Node.js.

PubNub Staff

PubNub Staff