Auto Moderation

The Auto Moderation module is an add-on to the Channel Monitor that lets you automate how you moderate messages in selected chat conversations.

Instead of monitoring messages one by one, you can automate this process by using Auto Moderation Functions.

Auto Moderation lets you follow a configurable wizard to create Functions that will run for each message on the selected keyset and channels.

Such Functions can perform the following operations:

• Delete selected messages before they are published on the channel or report such messages to the app admin (Spam detection (AI) Function powered by the PubNub-hosted AI model)
• Mask selected words you predefine as forbidden (Word masking Function)

Functions dependency & limitations

Auto Moderation relies on Functions v2 and is only available for Functions v2 users. Free tier covers a limited number of executions per month. If you exceed the limit, you'll need to upgrade to maintain service continuity.

Create configuration

To use Auto Moderation, start by defining the Function's configuration:

1. Log into the Admin Portal.
2. Go to the BizOps Workspace section in the left navigation panel.
3. Select Auto Moderation.

If it's the first configuration you define, you'll see the empty page when you open the Auto Moderation module.

Click the Create configuration button to open the setup wizard.

Once the wizard opens up, enter configuration name and description.

Select moderation behavior

In the next step, you can decide what type(s) of Auto Moderation Function(s) you want to apply to messages on your app's channels.

Spam detection (AI)

In this Auto Moderation function type, you can choose how you want AI to treat messages in selected channels on your keyset when it assesses them as spam.

A spam message may be either deleted from the channel or reported to the chat moderator for further actions.

To proceed, you must agree to the PubNub beta license agreement.

Handle blocked messages

Both Auto Moderation Functions (Spam detection (AI) and Word masking) are of the Before Publish or Fire type which means they check the message before it is published on a channel. That's why, if you decide to delete a message that is considered spam, this message will never appear in the Channel Monitor and your app won't be able to send it through PubNub as it encounters an error. In this case, you need to make sure you handle the returned error on your client in the app's UI to inform the app users about the applied moderation.

Message content path

By default, Auto Moderation Functions look for the message content to moderate under the text key in each message payload JSON structure. However, if your message structure has a different format, you can point Auto Moderation to the correct path in your messages:

Different message content paths

Unlike word lists, Auto Moderation and Channel Monitor don't share message path configuration which means you can specify different paths in both BizOps Workspace modules and these settings won't affect each other.

Word masking

In this Auto Moderation Function type, define which words you want to mask in messages published on all selected channels.

Word masking limitations

Word masking works only on initial publish, meaning that it masks words before they are published on channels. However, masked words can later be modified by message actions, such as editing the published message.

If you choose this Auto Moderation Function type, you must define a list of words (or choose an existing one) you want Auto Moderation Function to be on the lookout for.

When such a word is detected in a message, each letter in this word will be masked with an asterisk (*) character.

Shared word lists

Auto Moderation and Channel Monitor share word lists. This means you can reuse a word list previously created in Channel Monitor (either active or inactive) and apply it in the Auto Moderation configuration. You can also do the other way round, but remember to activate the word list in Channel Monitor for it to work.

Select keyset

On the next configuration wizard page, choose (a) keyset(s) on which you want the Auto Moderation Function(s) to run and press Next to proceed.

Select channels

Select channels on which you want Auto Moderation Functions to run for all messages and detect spam or mask forbidden words.

If you want to choose a large number of channels and not exceed the Deployments limit (50), specify channels using wildcards.

Test configuration

When you've completed the setup, you can check whether your configuration will work when activated.

To do that, click Test moderation behavior to see moderation running on the three test messages or enter and send a sample text message (JSON object, array, or literal) that either includes a word from the previously defined word list or should be considered spam.

Mocked publish call

This testing option only lets you locally check if the Auto Moderation mechanism works, but it does not publish an actual message on any of the channels within the selected keyset(s).

Check in the console if Auto Moderation worked as expected. Auto-moderated messages should be assigned proper labels (Message deleted or Message reported and Words masked).

You can also test your saved/activated configuration from the general Auto Moderation view once it's activated.

Save and activate configuration

When you're done configuring and testing Auto Moderation, you can save the settings (Save) or activate the configuration straightaway (Save and activate).

You'll see the active Auto Moderation Function and the dashboard showing statistics for the moderated messages in all active Functions.

Only an activated configuration deploys an Auto Moderation Function of type BizOps (as seen from the Functions module) which runs Auto Moderation on the selected channels within the keyset. Each activated configuration creates one Package with one Deployment and Functions in a number equal to the number of channels selected during the configuration setup.

Every word from the word list included in a message published on the selected channels within the scope of the given keyset will now be masked with asterisks.

Every spam message will be either deleted before it is published on a channel or reported to an admin for further actions.

Moderation paused during service outages

In case of any Function-related service disruptions (like communication errors between services or the LLM model issues), Auto Moderation will publish all messages instead of performing any moderation on them.

Edit configuration

An activated Auto Moderation can still be modified, but only from the Auto Moderation section in BizOps, not from the Functions module in the Admin Portal.

You can only have one configuration running on a given channel.

Activating an edited configuration will automatically restart the previously running one. However, if you want to activate a new configuration on the same channel, you will have to manually stop the previously running one.