Auto Moderation

Beta feature

This feature is currently in beta and available upon request. It may be subject to change and is not recommended for production use without testing. To get access to Auto Moderation, contact PubNub Support or Sales.

The Auto Moderation module is an add-on to the Channel Monitor that lets you automate how you moderate messages in selected chat conversations.

Instead of monitoring messages one by one, you can automate this process by using Auto Moderation Functions.

Auto Moderation lets you follow a configurable wizard to create Functions that will run for each message on the selected keyset and channels.

Such Functions can perform the following operations:

• Block selected messages from being published on the channel or publish but report such messages to the app admin (Spam detection (AI) Function powered by the PubNub-hosted AI model)
• Mask selected words you predefine as forbidden (Word masking Function)

Functions dependency & limitations

Auto Moderation relies on Functions v2 and is only available for Functions v2 users. Free tier covers a limited number of executions per month. If you exceed the limit, you'll need to upgrade to maintain service continuity.

Create configuration

To use Auto Moderation, start by defining the Function's configuration:

1. Log in to the Admin Portal.

2. Go to the BizOps Workspace section in the left navigation panel.

3. Select Auto Moderation.

   If it's the first configuration you define, you'll see the empty page when you open the Auto Moderation module.

4. Click the Create configuration button to open the setup wizard.

   

   Auto Moderation with an existing Before Publish Function

   Auto Moderation supports an edge case in which you already have a Before Publish or Fire Function running on a given channel and would like to enable moderation on it. The standard configuration wizard won't let you activate another Before Publish or Fire Function on the same channel (all Auto Moderation Functions are of such a type). In such a case, follow an alternative path in the configuration wizard by choosing the Generate code for Functions option. Go to the Auto Moderation with an existing Before Publish Function section for detailed steps.

5. Once the wizard opens, enter a configuration name and description.

   

Select moderation behavior

In the next step, you can decide what type(s) of Auto Moderation Function(s) you want to apply to messages on your app's channels.

Spam detection (AI)

In this Auto Moderation function type, you can choose how you want AI to treat messages in selected channels on your keyset when it assesses them as spam.

A spam message may be either blocked from being published on the channel or reported to the chat moderator for further actions.

To proceed, you must agree to the PubNub beta license agreement.

Handle blocked messages

Both Auto Moderation Functions (Spam detection (AI) and Word masking) are of the Before Publish or Fire type which means they check the message before it is published on a channel. That's why, if you decide to block a message that is considered spam, this message will never appear in the Channel Monitor and your app won't be able to send it through PubNub as it encounters an error. In this case, you need to make sure you handle the returned error on your client in the app's UI to inform the app users about the applied moderation.

Message content path

By default, Auto Moderation Functions look for the message content to moderate under the text key in each message payload JSON structure. However, if your message structure has a different format, you can point Auto Moderation to the correct path in your messages:

Different message content paths

Unlike word lists, Auto Moderation and Channel Monitor don't share message path configuration which means you can specify different paths in both BizOps Workspace modules and these settings won't affect each other.

Word masking

In this Auto Moderation Function type, define which words you want to mask in messages published on all selected channels.

Word masking limitations

Word masking works only on initial publish, meaning that it masks words before they are published on channels. However, masked words can later be modified by message actions, such as editing the published message.

If you choose this Auto Moderation Function type, you must define a list of words (or choose an existing one) you want the Auto Moderation Function to be on the lookout for.

When such a word is detected in a message, each letter in this word will be masked with an asterisk (*) character.

Shared word lists

Auto Moderation and Channel Monitor share word lists. This means you can reuse a word list previously created in Channel Monitor (either active or inactive) and apply it in the Auto Moderation configuration. You can also do the other way round, but remember to activate the word list in Channel Monitor for it to work.

Test configuration

In the next step you can check whether your configuration will work when activated.

To do that:

1. Click Test moderation behavior to see moderation running on the three test messages or enter and send a sample text message (JSON object, array, or literal) that either includes a word from the previously defined word list or should be considered spam.

   

   Mocked publish call

   This testing option only lets you locally check if the Auto Moderation mechanism works, but it does not publish an actual message on any of the channels within the selected keyset.

2. Check in the console if Auto Moderation worked as expected. Auto moderated messages should be assigned proper labels (Message blocked or Message reported and Words masked).

   

   You can also test your saved/activated configuration from the general Auto Moderation view once it's activated.

Select keyset

On the next configuration wizard page:

1. Choose a keyset on which you want the Auto Moderation Function(s) to run.
2. Press Next to proceed.

Select channels

Select channels on which you want Auto Moderation Functions to run for all messages and detect spam or mask forbidden words.

If you want to choose a large number of channels and not exceed the deployment limit (50), specify channels using wildcards.

If you notice the message below when trying to select a channel on which a Function should be deployed, this means you already have a Before Publish or Fire Function running on this channel and cannot deploy another.

In such a case, select the Generate code for Functions option to follow the alternative path in which you create moderation configuration but add it to the existing Function on your own.

Save and activate configuration

When you're done configuring and testing Auto Moderation, you can save the settings (Save) or activate the configuration straightaway (Save and activate).

You'll see the active Auto Moderation Function and the dashboard showing statistics for the moderated messages in all active and inactive Functions.

Only an activated configuration deploys an Auto Moderation Function of type BizOps (as seen in the Functions module) which runs Auto Moderation on the selected channels within the keyset. Each activated configuration creates one Package with one Deployment and a number of Functions equal to the number of channels selected during the configuration setup.

Every word from the word list included in a message published on the selected channels within the scope of the given keyset will now be masked with asterisks.

Every spam message will be either blocked before it is published on a channel or reported to an admin for further actions.

Provide feedback on auto moderated messages

You can help us improve the mechanism behind auto moderation by reviewing messages that were either blocked or reported. Read the Review auto moderated messages section for details.

The reported message will still be visible on the message list on the channel but marked with the Reported label.

Moderation paused during service outages

In case of any Function-related service disruptions (like communication errors between services or the LLM model issues), Auto Moderation will publish all messages instead of performing any moderation on them.

Edit configuration

An activated Auto Moderation can still be modified, but only from the Auto Moderation section in BizOps, not from the Functions module in the Admin Portal.

You can only have one configuration running on a given channel.

Activating an edited configuration will automatically restart the previously running one. However, if you want to activate a new configuration on the same channel, you will have to manually stop the previously running one.

Auto Moderation with an existing Before Publish Function

Auto Moderation supports an edge case in which you already have a Before Publish or Fire Function running on a given channel and would like to enable moderation on it.

Follow these steps:

1. Select Generate code for Functions in the initial Create configuration step.

2. Select moderation behavior.

3. Test configuration.

4. Select a keyset and save configuration.

5. Copy the moderation code from the pop-up modal.

   

   Copied code with explanation:

   // Imports the UGC moderation module. For more details, read the docs: https://www.pubnub.com/docs/serverless/functions/functions-apis/ugc-module
   const ugc = require('ugc');
   
   // Entry point for the Function; async so we can await the moderation call
   export default async (request) => {
       try {
           // Calls the moderation service with context about this publish
           const moderationResult = await ugc.moderateMessage({
               // ID of your Auto Moderation configuration
               configId: '0346b6d9-2183-4de3-ae29-1d98cb5e9784',
               // Full message payload the client sent
               message: request.message,
               // Publisher's UUID from the request
               userId: request.params.uuid,
               // Target channel for this publish

   show all 48 lines

6. Go to the Functions module on the Admin Portal and select the active Before Publish or Fire Function to which you want to add moderation configuration.

   

7. Add the previously copied moderation configuration code to the Function's body. The way you modify the existing Function's code differs depending on the Function logic.

   For example, this is an original body of a Function that tags detected @mentions in meta so downstream code/clients can notify mentioned users, highlight mentions, or apply mention-specific rules—without re-parsing the text:

   // Before Publish Function entry point; runs before the message is delivered.
   // You can modify `request.message` or `request.params.meta` here.
   export default (request) => {
       // Regex to detect @mentions like @alice or @user_123 (case-insensitive)
       const mentionPattern = /\B@[a-z0-9_-]+/gi;
   
       // Normalize the incoming message to a text string:
       // - if it's a string, use it directly
       // - if it's an object, try to read the `text` field
       // - otherwise, default to an empty string
       const text =
           typeof request.message === 'string'
           ? request.message
           : (request.message && request.message.text) || '';
   

   show all 37 lines

   Code with embedded moderation configuration:

   const ugc = require('ugc');
   
   export default async (request) => {
       // Parse existing meta (string → object)
       let meta = {};
       try {
           meta = typeof request.params.meta === 'string'
           ? JSON.parse(request.params.meta)
           : (request.params.meta || {});
       } catch (_) {
           meta = {};
       }
   
       // Run moderation (block or apply transforms)
       try {

   show all 52 lines

   Now this Function moderates via your configId (blocks or applies transforms), extracts @mentions, updates request.params.meta with mentions and hasMentions, then publishes the message unless blocked.

8. Save the changes (as a new revision) and redeploy the Function.