Schemas and validation in DataSync

Configuring a class is how you turn free-form payloads into data DataSync can guard and work with. Property definitions reject bad writes before they're stored, make fields searchable, and control who reads or edits each field. A class's TTL setting expires stale objects automatically, and adding a new class version lets you evolve your types over time while existing instances keep the validation, filtering, and access behavior of the version they were created against.

Entities and relationships are instances of the classes you define here. See Data model.

Classes type entities and relationships

Classes and versions

Classes are where you decide how strict your data is. A class can be a bare name, with payloads stored as-is, or a class with declared properties, searchable fields, and field-level access through projections. Every entity class also has a TTL, defaulting to 31 days if you don't set one, and there's no way to turn expiry off.

Once you've declared a class with Create a new entity class or Create a new relationship class, or in the Admin Portal, you can create instances from anywhere: your team manages types centrally, while clients create and update instances at runtime. Instances are created with the Core REST API, Create entity and Create relationship, or with an SDK.

A class is a name plus an integer version. A class name can be up to 128 characters, must start with a letter, and can otherwise contain only letters, digits, hyphens, and underscores. Multiple versions of the same class can coexist, and version numbers don't have to be sequential. Listing or filtering instances without naming a version matches every version of the class, not just the latest one. See Finding data.

Each entity or relationship instance records the version it was created against, but that version isn't frozen. You can move an instance to a different version of the same class by sending a new entityClassVersion on a replace (Update entity), or a JSON Patch to /entityClassVersion (Patch entity). What can't change is entityClass (or relationshipClass) itself, so a user entity can't become a channel entity.

Versions let you evolve a type without breaking existing data. In Bob's marketplace, you could define Product version 1, then later add a warranty field in Product version 2. Existing Product version 1 instances keep working unchanged.

Both versions stay live, and each instance records which one it currently uses:

Entity classes and relationship classes share the same versioning model, but relationship classes add their own settings. An entity class has no cardinality, but does have TTL configuration (see Data expiry). A relationship class has no TTL configuration of its own (see Relationships for how a relationship's expiry is derived instead), but does declare the settings below.

Relationship class settings

A relationship class declares cardinality (one-to-one, one-to-many, or many-to-many, enforced when a relationship is created, see Data model).

A relationship class can also restrict which entity class is allowed on each side, through entityAClass and entityBClass. Leave a side unset and any entity class can be used there. In Bob's marketplace, ProductOwner sets entityAClass to User and entityBClass to Product, so an attempt to own a channel instead of a product is rejected rather than stored. A side restriction accepts subclasses of the named class as well as exact matches, so a MarketplaceUser satisfies an entityAClass of User.

Each side also takes an optional class level, entityAClassLevel and entityBClassLevel, which disambiguates a keyset-level class from a Global class of the same name. Left unset, a keyset-level class takes precedence over a Global one. The check is based on class name, class level, and the extends hierarchy only, class version isn't part of it, so a side restriction matches an entity no matter which version of the class it was created against.

Property definitions

A class can declare property definitions. A property definition marks one payload field for filtering and projection purposes, and validates that field on every write regardless of its filtering mode: a payload with a missing non-nullable property, a value of the wrong type, or an array or object where a scalar is expected is rejected with a 400. The filtering mode only controls whether the field is indexed for search, a property left at the default filtering: none is validated but not searchable. Each property definition has:

Attribute	Required	Purpose
name	Yes	The property's name
path	Yes	A JSON Pointer to the field, starting at the whole object, for example /payload/price
valueKind	Yes	string, number, boolean, date, or datetime. A payload value of a different type at this field's path is rejected
filtering	No	none, simple, or full. Defaults to none
isNullable	No	Whether the field may be missing or null. If false, a payload missing this field, or with it set to null, is rejected. Defaults to true
projections	No	Which named projections include this field, given as objects: [{"name": "__default__"}]. Defaults to __default__ alone. At least one is required, and a class may use at most 3 distinct projection names, each up to 64 characters

Four property names are reserved

id, createdAt, and updatedAt name fields every object already carries natively, so a class property can't use those names at all. status is the one exception, and only in the exact form {"name": "status", "path": "/status"}, which declares the native field so you can put it in a projection. Every other use of these four names fails class creation or update with a 400 (DS-0906).

You don't need to declare them to search on them. All four are always filterable and sortable, whichever tier the query runs in. Refer to Built-in fields.

Property paths start at the object, not the payload

A property path addresses the whole object, where payload sits alongside the system fields, so the price field on a Product is /payload/price, not /price. This is the same coordinate space partial updates use. Paths aren't restricted to /payload, and a path such as /status resolves to the system status field and is extracted normally. If a path matches no field, /price instead of /payload/price for example, a nullable property is skipped and never matches a filter. A non-nullable property fails the write with DS-0650 NON_NULLABLE_PROPERTY, so an incorrect non-nullable path can make every write for that class fail. Refer to Payloads.

A datetime value can be either epoch milliseconds or an ISO-8601 string, a date value must be an ISO-8601 date string, and a number is stored as a double.

One property declaration does three jobs at once. Declaring /payload/price on the Product class with a filtering mode of simple or full validates it on every write, makes it filterable, and scopes its field-level access, all from the same definition:

The filtering mode controls how DataSync stores that field for querying, which in turn decides what you can do with it later:

Mode	How the field is stored	What you can do with it
none	Not extracted for querying	Nothing. The declared field is neither filterable nor sortable
simple	Extracted into the strongly consistent query store	Strongly consistent filtering (filter_fast) and sorting
full	Extracted into the strongly consistent query store, and additionally indexed in the eventually consistent search store	Both filtering parameters (filter_fast and filter) and sorting

full is a superset of simple, not an alternative to it. A full property works with filter_fast as well, so choosing full never costs you strongly consistent filtering. It does mean the field is written to two stores on every update.

In filter and sort expressions you always use the property's name, never its path. So a property named price at path /payload/price is filtered as price < 100.

See Data operations for how the two filtering tiers behave at query time.

Declare properties before you need them

Property definitions apply forward-only. Data written before a property was declared on the class is not retroactively indexed for that property, so it will not show up in filters on that field until it is rewritten.

Property definitions also control projection membership, which is how field-level read and write access is scoped. See Projections for the full model.

Field-level read and write access is controlled by projections, declared per property. See Projections.

Property definitions declare projection membership

Both filter_fast and filter operate on the properties you declare on a class, plus the four built-in fields you never declare. See Data operations.

Declared properties power search

Class inheritance

Class inheritance is how you build new classes on top of existing ones.

Entity class inheritance

Entity classes support single inheritance, and only upward across class levels: a class can extend a class defined at a higher level, but not one at the same level. Because the classes you create are keyset-level, that means you can extend the Global User and Channel classes but not your own other classes. Extending a same-level class is rejected with a 400.

Inheritance merges parent properties into the subclass automatically. You only declare the properties you're adding, not the ones you're inheriting, and a subclass can't drop an inherited property, since it's merged in whether you declare it or not. If you do redeclare a property with the same name as one on the parent, it's checked for compatibility: it must keep the parent's path, valueKind, and isNullable, so a subclass can't repoint an inherited property, change its value kind, or change its nullability through a redeclaration.

filtering mode and projections are not compared, so a subclass is free to make an inherited field more or less filterable, or place it in different projections, than its parent does.

Relationship class inheritance

Relationship classes don't support inheritance. The built-in Membership class has no equivalent extension path: you can't declare additional searchable properties on it. You can still store arbitrary fields in a membership's payload, they just aren't filterable, since only declared properties are searchable.

Global class inheritance

Global classes, like the built-in User and Channel entity classes and the built-in Membership relationship class, are provided by PubNub and cannot be modified: you cannot add, change, or remove their property definitions. Attempting to modify a Global class returns a 403.

For User and Channel, you can add your own fields by defining a class on your keyset that references the Global class as its parent through an extends reference. The subclass name is up to you, it does not need to match the parent's. The parent's name and type properties are inherited automatically, you only declare the fields you're adding. In Bob's marketplace, a MarketplaceUser class extends the Global User class to add profile fields.

Inheritance also affects search: querying a parent class returns instances of its subclasses too. In Bob's marketplace, MarketplaceUser extends the Global User class, so searching for User instances returns marketplace users as well:

To add fields to users or channels, define a keyset-level class that extends the Global User or Channel class, rather than trying to modify the Global class itself. See Users, channels, and memberships.

Extending User or Channel

Data expiry (TTL)

Entity classes carry a config.ttlSec setting. The default is 2678400 seconds (31 days), the minimum is 0, and the maximum is about 315569260 seconds (roughly 10 years). There is no never-expire option. A ttlSec of 0 doesn't mean immediate expiry, because expiresAt rounds up to the start of the next whole UTC day.

A class created with extends and no config of its own inherits its parent's TTL instead of the 31-day default. A class extending the Global User class without its own config gets User's 30-day TTL, not 31 days.

The built-in Global User and Channel classes carry a TTL of 30 days, so a plain user or channel expires 30 days after it was created unless you create it against a subclass of your own with a different TTL.

The expiry mechanics that matter when you design a class:

• expiresAt is computed once, at creation, as the creation time plus the class TTL, rounded up to the start of the next whole UTC day.
• Updating an instance does not extend or refresh its expiresAt.
• There is no per-instance override of the class TTL.
• Once expiresAt passes, the object stops being readable.
• A relationship has no TTL configuration of its own. It expires at the earlier of its two linked entities' expiry times, computed when the relationship is created.

Plan for expiry

Choose a TTL that matches how long the data should live. For long-lived data such as user profiles, set a long TTL and keep the roughly 10-year maximum in mind. For short-lived data, a short TTL cleans up automatically. In Bob's marketplace, a flashSaleBanner class could use a TTL of one hour so expired banners disappear without a manual delete. Because expiresAt rounds up to the next whole UTC day, that banner doesn't disappear an hour after it was created, it disappears at the start of the next UTC day. Short TTLs give you day-granularity cleanup, not minute-granularity.

Managing classes

Classes, their property definitions, and event rules are managed with the Admin API, or in Admin Portal, not through the SDKs. Entity classes and relationship classes have separate endpoint sets:

Operation	Entity class	Relationship class
List	Get all entity class entries	Get all relationship classes
Read one	Get entity class by ID	Get relationship class by name and version
Create	Create a new entity class	Create a new relationship class
Replace	Update entity class with complete resource replacement	Update relationship class with complete resource replacement
Partial update	Not available	Not available
Delete	Delete entity class by ID	Delete relationship class by name and version

Updating a class version replaces its entire property-definition set, rather than merging into it. Partial class updates aren't implemented. Send every property you want the version to keep. Omitting properties deletes all of them.

A class version can also be deleted. Deleting one that another class extends is rejected with a 409, and Global class versions can't be deleted at all.

Some parts of a class are fixed when you create it, some you can revise later, and some matter on every request:

• Design time: cardinality, entityAClass and entityBClass, and the extends reference are fixed when you create the class version. A new extends on an update is ignored.
• Later: property definitions, the class description, and TTL can all be revised. Editing a live version affects every instance pointing at it, so evolving a type usually means adding a new version instead.
• Every filter query: filtering follows whichever properties were declared on the class version, using the filtering mode you set for each.