Answer: No, you do not have to store your data with PubNub, but doing so can give you access to extra features & capabilities.
Where should you store your data?
Many customers choose to store their messages, history & metadata with PubNub since having all your data in one place enables additional features and insights into that data.
Also, many customers choose to store their message data outside of PubNub, perhaps because they have a separate storage infrastructure or run analytics that depend on data from other systems.
Additionally, many customers choose the best of both worlds, combining PubNub’s secure storage for real-time data and extracting that data for offline analytics and marketing purposes.
PubNub is flexible. You do not have to store data with us to implement a production application with real-time messaging features. Still, if you choose to, you get access to additional features and benefits, which we can only provide if we understand your data in context. The choice is entirely yours, and the aim of this blog is to help you make an informed decision.
What can you do with PubNub if you do not store data with us?
A lot.
You can still Publish and Subscribe to send and receive messages in real-time with 15 points of presence worldwide. Your solution will still scale to any number of users or devices and communicate over channels and channel groups without issue. PubNub’s presence feature will notify you when users and devices come online or offline without needing to store anything.
Your solution can still benefit from our 99.999% uptime SLA and be secured against unauthorized access. You can continue to push messages to mobile devices without worrying about complex device setups.
If you need to extend your application with serverside logic, PubNub Functions provides serverless computing that can scale with your application - though we host the code, you can still use Functions without storing any data.
What can you do if you store your data with PubNub?
Even more.
Some things are just easier if you store your data with PubNub
PubNub will store your messages for later retrieval if you enable message persistence in the admin portal. You can also store “actions” associated with those messages, which you can think of as message metadata; for example, you could indicate that a user has reacted to a message or mark the message as part of a larger thread. Keeping all your messages and message metadata in the same place makes it far easier for developers. Consider loading the last messages in a chat conversation along with reactions and unread message counts or retrieving the last sensor readings received from a remote device; that historical data can be retrieved with just one or two API calls.
If you want to share files or large chunks of data between users or devices, then PubNub offers a file-sharing API supported by all our SDKs. By enabling File Sharing in the admin portal, you can choose a custom retention period, which can be as short as one day, and then publish the file to any number of recipients in the same way as publishing a standard message. You do not have to store the file in PubNub to exchange it, and you could just as easily store the file elsewhere and send a link to that file with a PubNub message; however, you would then need to manage all the file access rights yourself rather than relying on your existing implementation of PubNub Access Manager.
With PubNub’s App Context, you can also store metadata about channels, users, and the relationships between channels and users (memberships). Let’s look at the user metadata example; you can store any custom data you like to define the user, such as a nickname, email address, and avatar. Some customers are happy to connect to an identity provider and store all their user data in PubNub. You can register to receive events whenever App Context data changes (for example, when a remote user changes their avatar or nickname, other clients can be notified about this so they can update their UI in real-time), so some PubNub customers only choose to store data within App Context that is likely to change. Still, however you decide to architect your application, PubNub is flexible enough to accommodate your needs. Channel and membership metadata work similarly - you could assign a description or custom state to a channel and some custom property like a ‘star’ to a membership.
Additional features that are only possible if you store your data with PubNub
Storing your data in PubNub can give you visibility into that data that would otherwise not be possible.
PubNub Insights provides dashboards that let you quickly drill into the most active channels and users over various time periods, enabling you to see where messages and users are coming from down to the city level. With these dashboards, you can instantly analyze how your app is being used, identify issues, and plan for the future.
PubNub metrics are split into channels, users, and messages.
The sort of questions the PubNub Insights dashboards can answer:
What is my application adoption rate?
Where are my users located?
At what time are users most active on my app?
Who are my most active users?
How to Extract Your Data from PubNub
The recommended way to send real-time PubNub data to a third-party system (e.g., for storage, analytics, or machine learning) is to use Events & Actions [E&A]).
E&A does not require any code to be written and is accessible from the PubNub admin portal.
Firstly, define an event associated with the data you want to send, for example, when a message is published. You can also associate a filter with that event, such as only firing when the message is published to a specific channel or sender.
Next, configure the action which will be triggered after the event fires. Which action you choose will depend on where you want your data to be stored, and Events & Actions currently supports the following actions:
Webhook: to integrate with other third-party servers in a fully event-driven manner.
Amazon SQS: To send data to Amazon’s message queuing service
Amazon Kinesis: To send data to Amazon’s data streaming service that handles large to extra-large data sets.
S3: To upload data to Amazon S3 in batches
One common question we receive is when you should use E&A and when you should use PubNub Functions. Since the latter can execute a serverless function after a message is published, it could easily send data to an external service. I put the following guide together to help answer that question: Events & Actions or Functions: Picking the right tool.
Conclusion
Wherever you want to store your data, PubNub is flexible enough for your use case; however, I usually make the following recommendations to customers:
Any data that can be updated in real time should be stored in PubNub, and your solution should take advantage of events to ensure those updates are notified to all interested parties.
Any data you need to retrieve quickly, such as the history of a chat conversation or a sensor reading over the past 7 days, should be stored in PubNub and retrieved using one of the PubNub SDK APIs for persistence.
Retain at least 3 months of data with us (but ideally more) and take advantage of PubNub Insights to better understand your customer behavior
Forward all your data to an external service using PubNub’s Events & Actions for later search, analysis, or marketing efforts.