Configure Auto Moderation

This guide shows you how to create an Auto Moderation policy, configure moderation features, and activate it on selected channels.

For an overview of how Auto Moderation works and how it compares to Channel Monitor, see Moderation.

Before you start

Before starting, confirm you have:

• A PubNub account on a paid plan. Auto Moderation runs as a Function on your keyset.
• Auto Moderation enabled on your account. Contact PubNub Support or Sales to request access.
• Access to the Admin Portal.
• The keyset and channel names where you want to apply moderation.

Auto Moderation is in Beta

Activating a policy requires you to accept the PubNub beta license terms of service.

Create a policy

1. Log in to the Admin Portal.
2. In the left sidebar, go to BizOps Workspace.
3. Select Auto Moderation.
4. Click Create configuration or Create policy (the label depends on whether you have existing policies).
5. Enter a policy name and description at the top of the policy editor.

Existing Before Publish Function

If a Before Publish or Fire Function already runs on a channel, you cannot activate another of the same type. In the activation step, choose Generate code for Functions instead of PubNub managed.

Select and configure features

In the Select features sidebar, enable the features you want. Click a feature to enable it and open its configuration panel. Use the toggle in the panel to disable a feature.

Spam detection

Spam detection analyzes English-language messages using AI and flags promotional spam and scam content.

1. Click Spam detection in the sidebar to enable it.
2. Choose the action to take when a message is flagged:
   • Block: the message is not published. The publisher receives an error. Because the message never reaches the channel, it doesn't appear in Channel Monitor's chat view. Moderators can still review it under Review moderated messages in Channel Monitor.
   • Report: the message is published but marked as Reported in Channel Monitor, and it's listed under Review moderated messages for manual review.
3. Agree to the PubNub beta license terms of service to proceed.

Spam detection uses an AI model to flag likely spam and scam content. Like any classifier, it can miss violations (false negatives) or flag legitimate messages (false positives). It does not guarantee perfect detection.

Profanity filter

The Auto Moderation profanity filter analyzes messages across 15 supported languages. It blocks or reports messages that exceed your toxicity thresholds.

1. Click Profanity filter in the sidebar to enable it.

2. Choose the action to take when a message is flagged:

   • Block: the message is not published. The publisher receives an error. Because the message never reaches the channel, it doesn't appear in Channel Monitor's chat view. Moderators can still review it under Review moderated messages in Channel Monitor.
   • Report: the message is published but marked as Reported in Channel Monitor, and it's listed under Review moderated messages for manual review.

3. Under Languages, select the languages the filter should analyze.

4. Under Thresholds, set the detection sensitivity for each category using the OFF / LOW / HIGH slider:

   Category	What it detects
   Toxicity	Hostile, abusive, or harmful language
   Harassment	Targeted insults or abusive behavior toward individuals
   Hate speech	Slurs or attacks based on identity
   Sexual content	Explicit adult content or pornography
   Spam	Repetitive, promotional, or deceptive content
   Profanity	Offensive or vulgar language
   Self-harm	Content involving self-injury or encouragement of self-harm
   Doxxing	Sharing or threatening to expose private personal information
   Scam	Fraudulent schemes or attempts to steal money or information
   Impersonation	Pretending to be another person or organization
   Raiding	Coordinated efforts to harass or overwhelm a community
   Political abuse	Extremist rhetoric, radicalization, or political propaganda

5. Agree to the PubNub beta license terms of service to proceed.

Word masking

Word masking replaces words from a configured list with asterisks before the message reaches subscribers.

1. Click Word masking in the sidebar to enable it.
2. Select Mask word as the action.
3. Choose an existing word list or create a new one:
   • To create a new list, give it a name, then add words or regular expressions to the Restricted words (patterns) field, one entry per line.
   • To import a list from a file, drag and drop a .txt or .csv file into the import panel, or click Select file. A word-masking list import file is limited to 32 KB.
   • To populate the list with a built-in set of common profanity patterns including leet-speak variations, click Generate.

Shared word lists

Auto Moderation and Channel Monitor share word lists. A list you create here is also available in Channel Monitor, and vice versa. Activate the list in Channel Monitor separately if you want it to highlight words there.

Word masking limitations

Word masking applies on initial publish only. Later message edits through message actions can change masked words.

Configure the message path

Auto Moderation checks the $.message.text field in your JSON payload by default. If your app uses a different message structure, update this path.

1. Click Configure message path in the top-right corner of the policy editor.
2. Enter the JSON path to the field that contains the message text, for example $.message.content.

This setting applies to all three features. Auto Moderation and Channel Monitor store message path configuration independently.

Test the policy

You can test the policy at any point before activating it.

1. Click Test policy in the top-right corner of the policy editor.
2. In the Test your policy panel, click a pre-built test message or enter your own and click Test.
3. Review the results. Each tested message shows the action taken and which feature triggered it. If multiple features trigger, the most severe action applies.

Testing is local only

Testing checks your policy configuration without publishing a real message to any channel.

Save or activate the policy

Use the buttons at the bottom of the policy editor:

• Create policy: saves the policy without activating it. From the Moderation policies overview, you can activate any saved policy later by clicking Activate policy next to it.
• Create & activate policy: saves the policy and opens the activation wizard.

Activate the policy

The activation wizard opens whether you click Create & activate policy or Activate policy from the overview. Choose a deployment method.

Deploy as PubNub managed

Use this option when no Before Publish Function is already running on the target channels. PubNub deploys and manages the moderation logic automatically.

1. Click Select keyset and choose a keyset. You can search by app, keyset, or subkey.

2. Select one or more channels from the keyset.

   To cover many channels without exceeding the 30-deployment limit, add a Channel pattern. A channel pattern counts as one deployment.

3. To add channels from another keyset, click the keyset selector, choose the next keyset, and select its channels. Repeat as needed.

4. Click Deploy policy.

After deployment, the policy detail page shows each keyset, app, channel count, and deployment status under Channel deployments.

Generate code for Functions

Use this option when a Before Publish or Fire Function already runs on a channel. PubNub generates moderation code for you to add to the existing Function.

1. Click Select keysets and choose the keyset.

2. Click Generate code for Functions.

3. Copy the generated code from the panel. The generated code calls ugc.moderateMessage with your policy's configId and handles blocking, reporting, and word masking:

   1const ugc = require('ugc');
   2
   
   3export default async (request) => {
   4    try {
   5        const moderationResult = await ugc.moderateMessage({
   6            configId: '<YOUR_CONFIG_ID>',
   7            message: request.message,
   8            userId: request.params.uuid,
   9            channel: request.channels[0],
   10        });
   11
   
   12        if (moderationResult.flagged && moderationResult.actions.includes('block')) {
   13            return request.abort('Moderated');
   14        }
   15
   
   16        if (moderationResult.flagged && moderationResult.actions.includes('report')) {
   17            // reporting is handled externally; add custom logic here if needed
   18        }
   19
   
   20        if (moderationResult.transform) {
   21            if (moderationResult.transform.message) {
   22                request.message = moderationResult.transform.message;
   23            }
   24            if (moderationResult.transform.meta) {
   25                const existingMeta = JSON.parse(request.params.meta || '{}');
   26                const transformedMeta = { ...existingMeta, ...moderationResult.transform.meta };
   27                request.params.meta = JSON.stringify(transformedMeta);
   28            }
   29        }
   30
   
   31        return request.ok();
   32    } catch(err) {
   33        console.log('Moderation Error:', err);
   34        return request.ok();
   35    }
   36};

   show all 36 lines

   The catch block runs when the call to ugc.moderateMessage itself fails, for example if the moderation service is unreachable or times out. As generated, that block logs the error and calls request.ok(), which publishes the message unmoderated. Failing open is Auto Moderation's intended default, in this code and in PubNub managed deployments. It keeps a problem in the moderation service from stopping messages across your channels.

   If your application needs fail-closed behavior instead, that is, dropping the message rather than risking an unmoderated delivery when the moderation check itself fails, deploy with Generate code for Functions and replace the catch block. A PubNub managed deployment can't be changed to fail closed.

       } catch(err) {
           console.log('Moderation Error:', err);
           return request.abort('Moderation unavailable');
       }

   Fail-closed has a tradeoff: if the moderation service is unreachable, no message publishes on that channel until it recovers. Pick the behavior that matches your application's risk tolerance.

4. In the Admin Portal, open the Functions module and select the active Before Publish or Fire Function on the target channel.

5. Add the moderation code to the Function body. Save the changes as a new revision and redeploy.

Resolve policy conflicts

Auto Moderation validates channels when you create or edit a policy to check for conflicts:

• Internal conflict: an active Auto Moderation policy is already running on the selected channel. Stop the existing policy before activating a new one on that channel.
• External conflict: an active Before Publish Function (not from Auto Moderation) is already running on the channel. Use Generate code for Functions to add moderation to that Function instead of deploying a separate one.

When a conflict exists, the UI shows a banner and per-row badges with resolution instructions.

Edit or delete a policy

From the Moderation policies overview, click the ... menu next to a policy and select Edit or Delete. You can edit a policy only from the Auto Moderation section in BizOps, not from the Functions module.

Activating an edited policy automatically restarts the previously running one on the same channel.

Related tasks

• Moderation overview. Understand when to use Auto Moderation versus Channel Monitor.