Storage API for PubNub Node.js SDK
The PubNub Storage and Playback Service provides real-time access to history for all messages published to PubNub. Each published message is timestamped to the nearest 10 nanoseconds, and is stored across multiple availability zones in several geographical data center locations. Stored messages can be encrypted with AES-256 message encryption, ensuring that they are not readable while stored on PubNub's network.
Messages can be stored for a configurable duration or forever, as controlled by the retention policy that is configured on your account. The following options are available: 1, 3, 5, 7, 15, or 30 days, and Forever.
Fetch History
Requires Storage & Playback add-onRequires that the Storage & Playback add-on is enabled for your key. See this page on enabling add-on features on your keys:
https://support.pubnub.com/hc/en-us/articles/360051974791-How-do-I-enable-add-on-features-for-my-keys-
Description
This function fetches historical messages from one or multiple channels. The includeMessageActions
flag also allows you to fetch message actions along with the messages.
It's possible to control how messages are returned and in what order.
- if you specify only the
start
parameter (withoutend
), you will receive messages that are older than thestart
timetoken - if you specify only the
end
parameter (withoutstart
), you will receive messages from thatend
timetoken and newer - if you specify values for both
start
andend
parameters, you will retrieve messages between those timetokens (inclusive of theend
value)
You will receive a maximum of 100 messages for a single channel or 25 messages for multiple channels (up to 500). If more messages meet the timetoken criteria, make iterative calls while adjusting the start
timetoken to fetch the entire list of messages from storage.
Method(s)
To run Fetch History
, you can use the following method(s) in the Node.js V4 SDK:
fetchMessages(
{Array channels, Number count, Boolean includeMeta, Boolean includeMessageType, Boolean includeUUID, Boolean includeMessageActions, String start, String end},
Function callback )
Parameter | Type | Required | Defaults | Description |
---|---|---|---|---|
Operation Arguments | Hash | Yes | A hash of arguments. | |
channels | Array | Yes | Specifies channels to return history messages from. Maximum of 500 channels are allowed. | |
count | Number | Optional | 100 | Specifies the number of historical messages to return per channel. Default/Maximum is 100 per channel . |
includeMessageType | Boolean | Optional | true | Pass true to receive the message type with each history message. |
includeUUID | Boolean | Optional | true | Pass true to receive the publisher uuid with each history message. |
includeMeta | Boolean | Optional | Whether message meta information should be fetched or not. | |
includeMessageActions | Boolean | Optional | Whether message added message actions should be fetched or not. Throws an exception in case if API called with more than one channel . Truncation will happen if number of actions on the messages returned is > 25000. Each message can have a maximum of 25000 actions attached to it. Consider the example of querying for 10 meesages. The first five messages have 5000 actions attached to each of them. The API will return the first 5 messages and all their 25000 actions. The response will also include a more link to get the remaining 5 messages. | |
start | String | Optional | Timetoken delimiting the start of time slice (exclusive) to pull messages from. | |
end | String | Optional | Timetoken delimiting the end of time slice (inclusive) to pull messages from. | |
callback | Function | Optional | Executes on a successful/unsuccessful fetchMessages . |
Truncated response
If you fetch messages with messages actions, the number of messages in the response may be truncated when internal limits are hit. If the response is truncated, a more
property will be returned with additional parameters. Send iterative calls to history adjusting the parameters to fetch more messages.
Basic Usage
Retrieve the last message on a channel:
// assuming pubnub is an initialized instance
// start, end, count are optional
pubnub.fetchMessages(
{
channels: ['ch1', 'ch2', 'ch3'],
start: "15343325214676133",
end: "15343325004275466",
count: 25
},
(status, response) => {
// handle response
}
);
Response
//Example of status
{
error: false,
operation: 'PNFetchMessagesOperation',
statusCode: 200
}
//Example of response
{
"channels":{
"my-channel":[
{
"message":"message_1",
"timetoken":"15483367794816642",
"uuid":"my-uuid",
"message_type":null
},
{
"message":"message_2",
"timetoken":"15483367810978163",
"uuid":"my-uuid",
"message_type":null
}
],
"my-channel-2":[
{
"message":"message_1",
"timetoken":"15483367827886667",
"uuid":"my-uuid",
"message_type":null
}
]
}
}
Other Examples
Fetch messages with metadata and actions
pubnub.fetchMessages( { channels: ['my_channel'], stringifiedTimeToken: true, includeMeta: true, includeMessageActions: true, }, function (status, response) { // handle status, response } );
Fetch messages with metadata and actions Response
// Example of status { "error": false, "operation": "PNFetchMessagesOperation", "statusCode": 200 } // Example of response { "channels":{ "my_channel":[ { "timetoken":"15741125155552287", "message":{ "text":"Hello world!", "uuid":"my-uuid", "message_type":null, "meta":"" }, "data":{ "reaction":{ "smiley_face":[ { "uuid":"my-uuid", "actionTimetoken":"15741125155943280" } ] } } } ] }, "more": { "url": "/v3/history-with-actions/sub-key/subscribeKey/channel/myChannel?start=15610547826970000&max=98", "start": "15610547826970000", "max": 98 } }
Delete Messages from History
Requires Storage & Playback add-on Requires that the Storage & Playback add-on is enabled for your key. See this page on enabling add-on features on your keys:
https://support.pubnub.com/hc/en-us/articles/360051974791-How-do-I-enable-add-on-features-for-my-keys-
Description
Removes the messages from the history of a specific channel.
Note
There is a setting to accept delete from history requests for a key, which you must enable by checking the Enable Delete-From-History
checkbox in the key settings for your key in the Admin Portal.
Requires Initialization with secret key.
Method(s)
To Delete Messages from History
you can use the following method(s) in the Node.js V4 SDK.
Note
The Delete Messages method behaves slightly differently than other history methods. Note that the start
parameter is exclusive and the end
parameter is inclusive.
deleteMessages({String channel, String start, String end},Function callback)
Parameter | Type | Required | Description |
---|---|---|---|
Operation Arguments | Hash | Yes | A hash of arguments. |
channel | String | Yes | Specifies channel messages to be deleted from history. |
start | String | Optional | Timetoken delimiting the start of time slice (exclusive) to delete messages from. |
end | String | Optional | Timetoken delimiting the end of time slice (inclusive) to delete messages from. |
callback | Function | Optional | Is called on successful or unsuccessful execution of deleteMessages . |
Basic Usage
pubnub.deleteMessages(
{
channel: 'ch1',
start: '15088506076921021',
end: '15088532035597390'
},
(result) => {
console.log(result);
}
);
Response
{
error: false,
operation: 'PNDeleteMessagesOperation',
statusCode: 200
}
Other Examples
Delete specific message from history
To delete a specific message, pass the
publish timetoken
(received from a successful publish) in theEnd
parameter andtimetoken +/- 1
in theStart
parameter. e.g. if15526611838554310
is thepublish timetoken
, pass15526611838554309
inStart
and15526611838554310
inEnd
parameters respectively as shown in the following code snippet.pubnub.deleteMessages( { channel: 'ch1', start: '15526611838554310', end: '15526611838554309' }, (result) => { console.log(result); } );
History (deprecated)
Requires Storage & Playback add-on Requires that the Storage & Playback add-on is enabled for your key. See this page on enabling add-on features on your keys:
https://support.pubnub.com/hc/en-us/articles/360051974791-How-do-I-enable-add-on-features-for-my-keys-
Note
This method is deprecated. Use fetch history instead.
Description
This function fetches historical messages of a channel.
It is possible to control how messages are returned and in what order, for example you can:
- Search for messages starting on the newest end of the timeline (default behavior -
reverse
=false
) - Search for messages from the oldest end of the timeline by setting
reverse
totrue
. - Page through results by providing a
start
ORend
timetoken. - Retrieve a slice of the time line by providing both a
start
ANDend
timetoken. - Limit the number of messages to a specific quantity using the
count
parameter.
Start & End parameter usage clarity:
If only the start
parameter is specified (without end
), you will receive messages that are older than and up to that start
timetoken value. If only the end
parameter is specified (without start
) you will receive messages that match that end
timetoken value and newer Specifying values for both start
and end
parameters will return messages between those timetoken values (inclusive on the end
value). Keep in mind that you will still receive a maximum of 100 messages even if there are more messages that meet the timetoken values. Iterative calls to history adjusting the start
timetoken is necessary to page through the full set of results if more than 100 messages meet the timetoken values.
Method(s)
To run History
you can use the following method(s) in the Node.js V4 SDK:
history( {String channel, Boolean reverse, Number count, Boolean stringifiedTimeToken, Boolean includeMeta, String start, String end}, Function callback )
Parameter | Type | Required | Defaults | Description |
---|---|---|---|---|
Operation Arguments | Hash | Yes | A hash of arguments. | |
channel | String | Yes | Specifies channel to return history messages from. | |
reverse | Boolean | Optional | false | Setting to true will traverse the time line in reverse starting with the oldest message first.If both start and end arguments are provided, reverse is ignored and messages are returned starting with the newest message . |
count | Number | Optional | 100 | Specifies the number of historical messages to return. Default/Maximum is 100 . |
stringifiedTimeToken | Boolean | Optional | false | If stringifiedTimeToken is specified as true , the SDK will return timetoken values as a strings instead of integers. Usage of setting is encouraged in javascript environments which perform round-up/down on large integers. |
includeMeta | Boolean | Optional | Whether message meta information should be fetched or not. | |
start | String | Optional | Timetoken delimiting the start of time slice (exclusive) to pull messages from. | |
end | String | Optional | Timetoken delimiting the end of time slice (inclusive) to pull messages from. | |
callback | Function | Optional | Executes on a successful/unsuccessful history . |
Using the reverse
parameter:
Messages are always returned sorted in ascending time direction from Storage/History regardless of reverse
. The reverse
direction matters when you have more than 100 (or count
, if it's set) messages in the time interval, in which case reverse
determines the end of the time interval from which it should start retrieving the messages.
Basic Usage
Retrieve the last 100 messages on a channel:
pubnub.history(
{
channel: 'history_channel',
count: 100, // how many items to fetch
stringifiedTimeToken: true, // false is the default
},
function (status, response) {
// handle status, response
}
);
Response
// Example of Status
{
error: false,
operation: "PNHistoryOperation",
statusCode: 200
}
// Example of Response
{
endTimeToken: "14867650866860159",
messages: [
{
timetoken: "14867650866860159",
entry: "[User 636] hello World"
},
{...},
{...},
{...}
],
startTimeToken: "14867650866860159"
}
Other Examples
pubnub.history(
{
channel: 'my_channel',
reverse: true, // Setting to true will traverse the time line in reverse starting with the oldest message first.
count: 3, // how many items to fetch
stringifiedTimeToken: true // false is the default
},
function (status, response) {
// handle status, response
}
);
- Use history() to retrieve messages newer than a given timetoken by paging from oldest message to newest message starting at a single point in time (exclusive):
pubnub.history(
{
channel: 'my_channel',
reverse: true, // Setting to true will traverse the time line in reverse starting with the oldest message first.
stringifiedTimeToken: true, // false is the default
start: '13406746780720711' // start timetoken to fetch
},
function (status, response) {
// handle status, response
}
);
- Use history() to retrieve messages until a given timetoken by paging from newest message to oldest message until a specific end point in time (inclusive):
pubnub.history(
{
channel: 'my_channel',
stringifiedTimeToken: true, // false is the default
end: '13406746780720711' // start timetoken to fetch
},
function (status, response) {
// handle status, response
}
);
Usage!
You can call the method by passing 0 or a valid timetoken as the argument.
getAllMessages = function(timetoken) {
pubnub.history(
{
channel: 'history_test',
stringifiedTimeToken: true, // false is the default
start: timetoken // start timetoken to fetch
},
function (status, response) {
var msgs = response.messages;
var start = response.startTimeToken;
var end = response.endTimeToken;
// if msgs were retrieved, do something useful with them
if (msgs != "undefined" && msgs.length > 0) {
console.log(msgs.length);
console.log("start : " + start);
console.log("end : " + end);
}
// if 100 msgs were retrieved, there might be more; call history again
if (msgs.length == 100)
getAllMessages(start);
}
);
}
//Usage examples:
//getAllMessages();
//getAllMessages(null);
pubnub.history({
channel: 'history_channel',
reverse: true, // Setting to true will traverse the time line in reverse starting with the oldest message first.
count: 100, // how many items to fetch
stringifiedTimeToken: true, // false is the default
start: '123123123123', // start timetoken to fetch
end: '123123123133' // end timetoken to fetch
}).then((response) => {
console.log(response)
}).catch((error) => {
console.log(error)
});
- Use history() to retrieve the three oldest messages by retrieving from the time line in reverse using Promises:
pubnub.history({
channel: 'my_channel',
reverse: true, // Setting to true will traverse the time line in reverse starting with the oldest message first.
count: 3, // how many items to fetch
stringifiedTimeToken: true // false is the default
}).then((response) => {
console.log(response)
}).catch((error) => {
console.log(error)
});
- Use history() to retrieve messages newer than a given timetoken by paging from oldest message to newest message starting at a single point in time (exclusive) using Promises:
pubnub.history({
channel: 'my_channel',
reverse: true, // Setting to true will traverse the time line in reverse starting with the oldest message first.
stringifiedTimeToken: true, // false is the default
start: '13406746780720711' // start timetoken to fetch
}).then((response) => {
console.log(response)
}).catch((error) => {
console.log(error)
});
- Use history() to retrieve messages until a given timetoken by paging from newest message to oldest message until a specific end point in time (inclusive) using Promises:
pubnub.history({
channel: 'my_channel',
stringifiedTimeToken: true, // false is the default
end: '13406746780720711' // start timetoken to fetch
}).then((response) => {
console.log(response)
}).catch((error) => {
console.log(error)
});
function getAllMessages(timetoken) {
pubnub.history(
{
channel: 'history_test',
stringifiedTimeToken: true, // false is the default
start: timetoken // start timetoken to fetch
}).then((response) => {
var msgs = response.messages;
var start = response.startTimeToken;
var end = response.endTimeToken;
// if msgs were retrieved, do something useful with them
if (msgs !== undefined && msgs.length > 0) {
console.log(msgs.length);
console.log("start : " + start);
console.log("end : " + end);
}
// if 100 msgs were retrieved, there might be more; call history again
if (msgs.length == 100) {
getAllMessages(start);
}
}).catch((error) => {
console.log(error);
});
}
//Usage examples:
//getAllMessages();
//getAllMessages(null);
pubnub.history(
{
channel: 'my_channel',
stringifiedTimeToken: true,
includeMeta: true,
},
function (status, response) {
// handle status, response
}
);