DOCS
Ask AI

Create a Kafka action

Create a Kafka action when you want Events & Actions to forward matching PubNub events to an Apache Kafka topic. You configure the action in the Admin Portal, and it connects to any correctly configured broker, self-hosted or managed, because Events & Actions is hosting-agnostic for Kafka.

If you already have a Kafka topic, broker address, and credentials, skip to Configure the Kafka action.

Set up your Kafka environment​

Gather the following information from your Kafka environment before you configure the action:

  • Topic and, optionally, a key.
  • Broker URL and port.
  • Username and password.

If you host Kafka yourself or on a managed service such as Amazon MSK, use the Kafka Quick Start or your provider's setup guide to create a cluster and topic. Then gather the information above.

If you use Confluent Cloud, follow these steps instead.

Create a Confluent Cloud cluster, topic, and API key​

  1. Log in to Confluent Cloud and create a Kafka cluster to receive PubNub messages. Select a cloud provider and cluster type, and complete the wizard.
  2. After the cluster launches, open Cluster Settings and note the Bootstrap server. This is the broker URL you enter in the Admin Portal.
  3. On the cluster overview page, click Topics, then Create topic, and configure it. Create the topic before you send PubNub data to it, so the first delivery doesn't fail against a topic that doesn't exist yet.
  4. On the cluster overview page, click API keys, then Create key, and configure it. Note the Key and Secret values before you click Download and continue. Clicking it hides the secret afterward, so copy both values first.

Configure the Kafka action​

  1. In Events & Actions in the Admin Portal, select your app and keyset, then click + Add Action.
  2. Select Apache Kafka as the action type.
  3. In Routing Key, enter your topic, for example topic_0.
  4. If you want to keep related events in order, add a key by entering topic:key, for example topic_0:stamford. Kafka guarantees order only within a single partition, and a key routes matching messages to the same partition consistently.
  5. In Authentication Mechanism, select the type your Kafka setup uses: SASL/PLAIN, SCRAM-SHA-256, or SCRAM-SHA-512.
  6. In Username and Password, enter your Kafka credentials. In Confluent Cloud, use the Key as the username and the Secret as the password.
  7. In Brokers, enter the broker URL, for example 574mf0rd-bridge.ldn-west:1905, or a comma-separated list such as hostname:port, hostname2:port. Use a secure listener because PubNub rejects unsecured connections to Kafka.
  8. If you want failed deliveries retried automatically, enable and configure Kafka retry. See Available actions for the retry policy and backoff formula.
  9. Click Add event listener and pair the action with an existing event listener, or create one.
  10. Click Save changes.

Publish a message that matches the paired listener's filter and confirm it arrives on the Kafka topic.

Was this page useful?

Last updated on