TwistedTwistedTornadoAsyncioNativePythonPython-Twisted V4 Storage & Playback Tutorial for Realtime Apps

 

NOTICE: Based on current web trends and our own usage data, PubNub's Python Twisted SDK is deprecated as of May 1, 2019. Deprecation means we will no longer be updating the Python Twisted SDK but will continue to support users currently using it. Please feel free to use our other Python SDK offerings as they will continue to be supported and maintained. If you would like to use the Python Twisted SDK specifically, we would love to work with you on keeping this project alive!

Requires that the Storage and Playback add-on is enabled for your key. How do I enable add-on features for my keys? - see http://www.pubnub.com/knowledge-base/discussion/644/how-do-i-enable-add-on-features-for-my-keys
PubNub's Storage and Playback feature enables developers to store messages as they are published, and retrieve them at a later time. Before using this feature it must be enabled in the PubNub Admin Console.
Being able to pull an archive of messages from storage has many applications:
  • Populate chat, collaboration and machine configuration on app load.
  • Store message streams with real-time data management.
  • Retrieve historical data streams for reporting, auditing and compliance (HIPAA, SOX, Data Protection Directive, and more).
  • Replay live events with delivery of real-time data during rebroadcasting.
As needed, specific messages can be marked "do not archive" when published to prevent archiving on a per-message basis, and storage retention time can range from 1 day to forever.
These code samples build off code defined in the Pub & Sub tutorial, so before proceeding, be sure you have completed the Pub & Sub tutorial first.
To begin, lets populate a new channel with some test publishes that we'll pull from storage using the history() method.
# if we're using inlineCallbacks
for i in range(0, 500):
    envelope = yield pubnub.publish()\
        .message(['message#', i])\
        .channel('history_channel')\
        .should_store(True)\
        .deferred()

    print("%d: %s" % (i, envelope.status.is_error()))

# if we're not using inlineCallbacks
for i in range(0, 500):
    def print_if_error(envelope):
        print("%d: %s" % (i, envelope.status.is_error()))

    d = pubnub.publish()\
        .message(['message#', i])\
        .channel('history_channel')\
        .should_store(True)\
        .deferred()

    d.addCallback(print_if_error)
In the above example, we subscribe to history_channel, and onConnect, we'll publish a barrage of test messages to it. You should see these messages as they are published on the console.
Now that we've populated storage, we can pull from storage using the history() method call:
envelope = yield pubnub.history()\
    .channel("history_channel")\
    .count(100)\
    .future()
The response format is
[
	["message1", "message2", "message3",... ],
	"Start Time Token",
	"End Time Token"
]
By default, history() returns the last 100 messages, first-in-first-out (FIFO). Although we specified 100 for the count, that is the default, and if we hadn't specified a count, we would have still received 100. Try this example again, but specify 5 for count to see what is returned.
Setting the reverse attribute to true will return the first 100 messages, first-in-first-out (FIFO). Try this example again, but specify true for reverse to see what is returned.
The timetoken response value is a string, representing 17-digit precision unix time (UTC). To convert PubNub's timetoken to Unix timestamp (seconds), divide the timetoken number by 10,000,000 (10^7).
Given this example, you will get the last two messages published to the channel:
# if we're using inlineCallbacks
envelope = yield pubnub.history()\
    .channel("history_channel")\
    .count(2)\
    .deferred()

# if we're not using inlineCallbacks
d = pubnub.history()\
    .channel("history_channel")\
    .count(2)\
    .deferred()

d.addCallback(my_callback)
[[498,499],14272454823679518,14272454845442942]
To page for the next 2, use the set the start attribute to start timetoken value, and request again with all other settings the same:
[[496,497],14272454791882580,14272454823679518]
As illustrated, paging with the default reverse as true pages 2 at-a-time starting from newest, in FIFO order. If you repeat these Paging example steps with reverse as false, you will page 2 at-a-time as well, starting instead from the oldest messages, but still in FIFO order. You will know you are at the end of history when the returned start timetoken is 0.

 
Usage!
You can call the method by passing 0 or a valid time token as the argument.
@inlineCallbacks
def get_all_messages(start_tt):
    envelope = yield pubnub.history()\
        .channel('channel')\
        .count(100)\
        .start(start_tt)\
        .deferred()

    msgs = envelope.result.messages
    start = envelope.result.start_timetoken
    end = envelope.result.end_timetoken
    count = len(msgs)

    if count > 0:
        print("%d" % count)
        print("start %d" % start)
        print("end %d" % end)

    if count == 100:
        get_all_messages(start)
To pull from a slice of time, just include both start and end attributes:
# if we're using inlineCallbacks
envelope = yield pubnub.history()\
    .channel("history_channel")\
    .count(100) \
    .start(13847168620721752)\
    .end(13847168819178600)\
    .deferred()

# if we're not using inlineCallbacks
d = pubnub.history()\
    .channel("history_channel")\
    .count(100) \
    .start(13847168620721752)\
    .end(13847168819178600)\
    .deferred()

d.addCallback(my_callback)
d = pubnub.history()\
    .channel("my_channel")\
    .count(100)\
    .include_tometoken()
    .deferred()
d.addCallback(my_callback)

Retrieves up to 100 messages starting with the message stored before the start timetoken parameter value and older. In other words, the retrieval is from newest to oldest or in descending time direction.

first msg ------------ start tt ------------ last msg
               <------]

Retrieves up to 100 messages starting with the message stored at the end timetoken parameter (if one exists with that timetoken) value and newer. In other words, the retrieval is from oldest to newest or in ascending time direction.

first msg ------------ end tt ------------ last msg
                      [------------>

Retrieves up to 100 messages starting with the message stored after the start timetoken parameter value and newer. In other words, the retrieval is from oldest to newest or in ascending time direction.

first msg ------------ start tt ------------ last msg
                               [-------->

Retrieves up to 100 messages starting with the message stored at the end timetoken (if one exists with that timetoken) parameter value and older. In other words, the retrieval is from newest to oldest or in descending time direction.

first msg ------------ end tt ------------ last msg
                <------------]

Retrieves up to 100 messages starting with the message stored at the end timetoken (if one exists with that timetoken) parameter value and older but not older than the message stored after the start timetoken. In other words, the retrieval is from newest to oldest or in descending time direction ending with the message that comes after (is older) than the start timetoken value.

first msg ----- start tt -------- end tt ---- last msg
                        [   <-----------]