Given all the promise of the Internet, we initially experienced it only as an information dump with millions of websites containing billions of web pages hyperlinked to one another. With Web 2.0 we envisaging a semantic web powered by dynamic web applications where information was stored and retrieved structurally. Now with the emergence of Web 3.0, we are witnessing yet another transformation of the Internet in the form of cognitive apps and human-machine interaction.
As applications become more engaging and interactive, so does the need for messaging APIs and a variety of techniques such as long polling, that enable the underlying communication mechanisms. As a developer, how do you adapt to these rapid strides in the evolution around internet and communication technologies?
This blog post will take you through the journey of exploring the various web messaging frameworks and technologies, both legacy and emerging, that will power the apps of today and tomorrow. We will also look at how the PubNub Data Stream Network leverages these technologies to enable developers like you to build the most reliable and scalable apps over the Internet.
HTTP (Hyper Text Transfer Protocol) was the first protocol that defined the specification for Internet communication during the Web 1.0 era. This was the way to transfer hypertext documents over the Internet, and request for external media resources linked to the documents for rendering in the browser.
HTTP follows a simple request-response pattern, wherein the client requests the hypertext document, addressed by an URL from the server and the server obliges by sending the document as a response to the client’s request.
Over the years, HTTP has been enhanced to bring in additional features to support newer applications requirements and security features (HTTPS). If you consider yourself a protocol nerd then you can check out the HTTP1.1 specification in IETF. The newer version, HTTP2 is available here.
REST is a methodology for creating HTTP/HTTPS enabled applications that adhere to a structured means of communication over the Internet. In a broader sense, REST mandates that the communication should happen through well-defined web services that define their own APIs (Application Programming Interface).
REST is a powerful concept and its emergence has led to the dominance of “as-a-Service” models of application delivery over the Internet vis-a-vis the traditional methods of installing applications on your local computer. If you use Google docs or spreadsheets then you are already using REST APIs to store and edit documents in the cloud. Or if are an outer space aficionado, then you might be interested to explore the NASA open APIs which give you access to NASA images of space and other planets.
To understand REST better, let’s take a step back in the history. Back in the days of Web 1.0 when websites were just a bunch of .html files, each webpage had an URL that was formed by combining the domain name followed by the file path of the HTML file. Users had to type in or click that URL to retrieve the web page. These web pages were the endpoints and the URLs were the APIs for website visitors to interact with the site.
As web applications became more engaging, the web servers evolved to define URLs for logical endpoints instead of the actual HTML files. These logical endpoints allowed client applications to engage with Web servers in a modular way, through a bunch of services.
Let’s consider a simple Todo app. The operations for this app can be structured as different services that operate on ToDo list, Todo items, and users. With REST, the server functions of the app should be arranged in a structured and modular way.
While the underlying protocol driven by HTTP/HTTPS remains the same, the REST APIs now becomes an embodiment of the end user application and its entities, namely the users, todo lists, and todo Items. This way REST provides a framework to think and define the application services in an abstract way. This leads to a more concrete and sensible implementation of the application functionality.
If you are curious, here is a light read on a brief history of the evolution of REST.
While REST provides a standardized structure to APIs, it does have some limitation. Consider the REST APIs for the ToDo app above. All the constituent entities of the app, namely the todo list, the todo item, and the users are linked in some way. That is, the users create todo lists which contain a bunch of to-do items. However, the REST APIs are designed to retrieve these entities separately. If you have to display all the todo lists of a user, then you have to make multiple API calls to retrieve the user and then retrieve the user’s todo lists.
GraphQL is an open standard that leverages the semantic association between the entities to retrieve multiple resources in a single query. It defines a query language and a runtime that can sit in the front of an API endpoint such that you can query and retrieve all the todo lists of a user in one shot.
And here is a quick tutorial on building REST endpoints with Functions. With this, you can write your own backend webhooks for handling the business logic of the todoApp.
You can also check out the quick guide on building a REST API in 5 minutes. If you are still wondering about the possibilities of using Functions as a REST webhook then check out this chat app which you can build with Functions in no time.
WebSockets bring a spark to web applications by enabling real-time, bi-directional internet communication between applications. Again, back to the Web 1.0 days, you can probably recall that the world wide web was all about fetching web pages from servers. That’s the reason why HTTP was designed as a request/response protocol.
With Web 2.0, we have a whole new way of interacting with websites, that have now turned into single page web applications. These applications are real-time, user-centric and highly engaging. Take the case of Facebook for example. HTTP can still be used for driving such web applications, however, it is inefficient for such real-time interactions.
WebSocket allows you to have a persistent bi-directional communication, instead of the frequent request/response message exchanges.
Metaphorically, we could think of this as an army with a chain of command. Let’s make the web server the General, and all the browsers as the soldiers on the ground waiting for orders. Using HTTP/REST, if every soldier has to ask the General if there are any new orders, it burdens the General tremendously, particularly when there isn’t anything new. That means the General is saying “no, nothing new” most of the time.
If your servers are the General, their resources are being wasted mostly saying that there is nothing new.
Using the same metaphor, WebSockets being connected are like each soldier having a radio, and when the General has a new order, he can broadcast that order into the radio. You can learn more about the technical details of WebSockets from the official W3C specification.
There are many implementations of WebSockets across programming languages. One of the popular libraries is Socket.IO. However, to build applications at scale and with a high degree of reliability, it is not just enough to have bi-directional communication between two ends.
At PubNub, we embrace scalability and real-time performance as a way of life. Hence we have wrapped this concept of WebSocket around our own data streaming technology that empowers you to scale out your apps without spending extra effort in development.
Publish/Subscribe is more of a paradigm or a communication pattern. With the advent of Web 3.0, we have entered into a new realm of collaborative applications that are no longer merely human-centric. We are now talking about human-to-machine and machine-to-machine interfaces because we are increasingly interacting with cyber-physical systems that are smart and have cognitive capabilities.
The scale of such interaction always fans out as a multi-party communication. That’s where we need something more than a mere request-response or bidirectional mechanism. The Publish/Subscribe approach provides a way out. Imagine, you have to send commands to multiple devices at once or a device should update its status to a web dashboard as well multiple app screens. All of this can be achieved seamlessly with Publish/Subscribe.
You can think of Publish/Subscribe as a bi-directional, multi, request/response system. Confused? Let take an analogy.
Think of your grandfather’s telephone. You could dial another person’s phone number and talk to him. But how difficult was it to bring in more people into the conversation? Now, think about Google Hangouts, or WhatsApp group. In essence, the communication is now driven by a virtual entity acting as a Hub, with end users acting as publishers or subscribers of messages.
A web messaging API that supports Publish/Subscribe mechanism, allows multi-party collaboration. It is a flexible way of communication and there already exist several standardized implementations of this. Take the case of MQTT, which is the preferred protocol used for IoT. In a typical IoT application, data coming from a sensor needs to be forked out to multiple places.
Publish/Subscribe mechanism provides the basis for achieving all this without the overhead of multiple request/response interactions. In this case, all the messages are orchestrated by a MQTT broker, which is the equivalent of the hub in the previous example.
With PubNub, you have a superior alternative in the form of global data stream network where a published message can be delivered to millions of subscribers within a tenth of a second. And unlike MQTT, PubNub also provides additional features such as message persistence, access management, and presence.
If you are building any collaborative application over the internet then PubNub ensures message delivery in real time and at a scale that can match any application requirement. Be it a public chat room, or an IoT network spanning millions of nodes, PubNub simply scales effortlessly by leveraging its data stream network.
The flexibility of Publish/Subscribe Paradigm along with PubNub’s promise of reliable and fast data messaging allows you to build diverse kinds of applications.
At PubNub, we have taken the Publish/Subscribe paradigm to a different level to make it truly flexible. This means that you can deploy your own arbitrary code as serverless functions that can operate upon the messages as they traverse from publishers to subscribers.
This capability is supported by Functions itself. So instead of configuring a Function as a REST webhook, you can also configure it as a serverless instance along the path of messages. This allows enormous capabilities in building serverless applications where the application business logic can be built by programming the network.
PubNub provides pre-built functions in the form of Blocks via the Blocks Catalog. Developers can use these blocks as a reference and build their own blocks for specific applications.
There are common underlying technologies for a dating app, and in this post, we’ll talk about the major technologies and designs...