Serialization

Introduction

PhysX 3 features two approaches to serialization:

  • API-level serialization to RepX (an XML format)
  • Binary serialization

API-level serialization uses a human readable XML format - RepX - that directly corresponds to the PhysX API. It is therefore suitable for manual inspection and modification for debugging purposes. It offers platform independence and further supports loading data that was serialized with a previous PhysX SDK version. API-level serialization is not expected to be used in performance critical situations.

The binary serialization approach on the other hand supports instantiation of PhysX objects directly from memory without copying data. This in-place deserialization method is well suited for performance critical real time situations. However, this approach is also less flexible as the binary format is specific to a given platform and PhysX SDK version. PhysX provides functionality to convert binary serialized data from authoring platforms to run-time platforms to ease the asset management.

Note

cooking also generates a binary output stream. The primary purpose of cooking, however, is to translate from a user format to a format suitable for the SDK runtime, and so it is not considered a serialization mechanism. Loading a cooked mesh from a stream involves allocation and endian conversion. As a consequence, it is much less efficient than PhysX' binary serialization mechanism. See Shapes for more details about cooking.

The following documentation will discuss how to use both serialization approaches. It will show how to build collections of PhysX objects and how these collections are serialized and deserialized. Further it will show how dependencies to other PhysX objects or application side objects can be re-established when deserializing.

PhysX also supports extending serialization to custom types, such as specialized joints. This is described in more detail in Section Extending Serialization.

First Code

The following code creates and serializes a rigid dynamic using both RepX and binary formats:

// Create a material, a shape and a rigid dynamic
PxSphereGeometry geometry(1.0f);
PxMaterial* material = PxGetPhysics().createMaterial(0.0f, 0.0f, 0.0f);
PxShape* shape = PxGetPhysics().createShape(geometry, *material);
PxTransform t = PxTransform(PxIdentity);
PxRigidDynamic* dynamic = PxCreateDynamic(PxGetPhysics(), t, geometry, *material, 1.0f);

PxSerializationRegistry* registry = PxSerialization::createSerializationRegistry(PxGetPhysics());

// Create a collection and all objects for serialization
PxCollection* collection = PxCreateCollection();
collection->add(*dynamic);
PxSerialization::complete(*collection, *registry);

// Serialize either to binary or RepX
PxDefaultFileOutputStream outStream("serialized.dat");

// Binary
    PxSerialization::serializeCollectionToBinary(outStream, *collection, *registry);
//~Binary

// RepX
    PxSerialization::serializeCollectionToXml(outStream, *collection, *registry);
//~RepX

Most operations related to serialization require an instance of PxSerializationRegistry, which provides information on how to serialize PhysX types. In order to serialize a PhysX object, it needs to be added to a PxCollection. If an object has dependencies on other PhysX objects, they need to be serialized as well. PxSerialization::complete adds all the required objects to the collection.

The following code deserializes the rigid dynamic and adds it to a scene for simulation:

PxSerializationRegistry* registry = PxSerialization::createSerializationRegistry(PxGetPhysics());

// Binary
    // Open file and get file size
    FILE* fp = fopen("serialized.dat", "rb");
    fseek(fp, 0, SEEK_END);
    unsigned fileSize = ftell(fp);
    fseek(fp, 0, SEEK_SET);

    // Allocate aligned memory, load data and deserialize
    void* memory = malloc(fileSize+PX_SERIAL_FILE_ALIGN);
    void* memory128 = (void*)((size_t(memory) + PX_SERIAL_FILE_ALIGN)&~(PX_SERIAL_FILE_ALIGN-1));
    fread(memory128, 1, fileSize, fp);
    fclose(fp);
    PxCollection* collection = PxSerialization::createCollectionFromBinary(memory128, *registry);
//~Binary

// RepX
    // Load file and deserialize collection - needs cooking library
    PxDefaultFileInputData inputData("serialized.dat");
    PxCollection* collection = PxSerialization::createCollectionFromXml(inputData, *cooking,
                                                                        *registry);
//~RepX

scene->addCollection(*collection);

When deserializing a binary serialized collection, the data first needs to be copied to a memory block that is aligned to 128 bytes. The memory block may not be deallocated before the objects have been released: it needs to persist for the entire lifetime of the objects. This does not apply to RepX deserialization, as the memory for the corresponding PhysX objects is allocated within PhysX. Finally the objects of the resulting collection can be added to the scene with PxScene::addCollection.

In-depth Discussion

Collections

The serialization system makes use of a class PxCollection, which manages references to objects deriving from PxBase. Each collection represents a set of objects. Collections maintain a mapping between IDs of type PxSerialObjectId and objects in the collection. IDs may be defined by the application. One caveat here is that the IDs must be unique within a collection, but do not have to be unique across different collections. If the latter is required by the application, it is the application's responsibility to ensure it.

Here is an example of how to iterate over a collection, for instance to ensure that the objects intended for serialization have all been added to the collection. When doing so PhysX' dynamic typing mechanism can be used to classify the objects:

PxCollection* collection;
PxU32 size = collection->getNbObjects();
for(PxU32 i=0; i<size; i++)
{
    PxBase* object = collection->getObject(i);
    if(!object->is<PxActor>())
        continue;

    switch((PxConcreteType)object->getConcreteType())
    {
    case PxConcreteType::eRIGID_DYNAMIC:
    ...
    }
}

Note

In order to simplify releasing object within a collection, PhysXExtensions contains a function to remove and release all objects from a collection: PxCollectionExt::releaseObjects.

Note

Releasing an object within a collection invalidates the mapping from indices to objects.

A collection is said to be complete if no contained objects depend on an object outside of the collection. For example, an actor, a shape with a box geometry, and the material of the shape would together form a complete collection. The same collection without the material would be incomplete.

../_images/Serialization_Complete.png

Figure 1: Left: Complete Collection, Right: Incomplete Collection

For a formal definition please refer to Complete.

Both complete and incomplete collections can be serialized, but when deserializing an incomplete collection, references to objects which were not serialized will need to be resolved. The following two sections describe how PhysX collections can be serialized and deserialized using the binary format or RepX. The first section shows how to deal with complete collections, and the second section shows how to deal with incomplete collections.

Serializing Complete Collections

This code snippet shows how to prepare a collection of PhysX objects for serialization (e.g. an actor, its shapes, and the materials and meshes they reference.):

PxPhysics* physics;                                         // The physics SDK object
PxRigidDynamic* dynamic = PxCreateDynamic(...);             // Create a rigid dynamic

                                                            //Create a serialization registry
PxSerializationRegistry* registry = PxSerialization::createSerializationRegistry(*physics);

PxCollection* collection = PxCreateCollection();            // Create a collection
collection->add(*dynamic);                                  // Add it to the collection

PxSerialization::complete(*collection, *registry);          // Adds all objects required to
                                                            // recreate the dynamic after
                                                            // deserialization

Instead of using PxSerialization::complete it is possible to manually add the objects required for serialization. All objects the PxRigidDynamic references would need to be added and then all objects referenced by the newly added objects would need to be added as well and so forth. See definitions: Requires, Complete.

By default PxSerialization::complete follows references from joints to their actors, but not from actors to their joints. The followJoint parameter can be used to change the behavior of PxSerialization::complete to add the joints attached to each actor. This will cause entire actor-joint chains to be added to the collection.

When all the necessary objects have been added to a collection, create an implementation of the PxOutputStream interface, then serialize the collection:

PxColletion* collection;                                  // Complete collection without orphans
PxSerializationRegistry* registry;                        // Registry for serializable types
PxOutputStream& outStream = ...;                          // Implemented by the application

// Serialize

// Binary
    PxSerialization::serializeCollectionToBinary(outStream, *collection, *registry);
//~Binary

// RepX
    PxSerialization::serializeCollectionToXml(outStream, *collection, *registry);
//~RepX

// Collection and registry can be released if they are no longer required.
// Note that releasing the collection will not release the contained objects!
collection->release();
registry->release();

Note

Serialization of objects in a scene that is simultaneously being simulated is not supported and leads to undefined behavior.

The following code shows how to deserialize a collection from a memory block or XML:

PxSerializationRegistry* registry;                        // Registry for serializable types
PxCooking* cooking;                                       // Cooking library needed for
                                                          // instantiating objects by RepX

// Deserialize

// Binary
    void* memory128 = ...;                                // A 128-byte aligned buffer previously
                                                          // loaded from disk by the user

    PxCollection* collection = PxSerialization::createCollectionFromBinary(memory128, *registry);
//~Binary

// RepX
    PxInputData& inputData = ...;                         // Implemented by the application
    PxCollection* collection = PxSerialization::createCollectionFromXml(inputData, *cooking,
                                                                        *registry);
//~RepX

To add all the objects to the scene and release the collection and registry:

PxScene* scene;                                           // The scene object
scene->addCollection(*collection);
collection->release();
registry->release();

See Serializable for the exact set of conditions a collection must satisfy in order to be serialized. These conditions can be checked with PxSerialization::isSerializable(...).

Serializing Incomplete Collections

Another common use case is where a collection of actors and joints - say, a rag doll - will be deserialized multiple times, with each instance sharing the same materials and meshes. To achieve this, serialize two collections:

  • a collection A of the materials and meshes that will be deserialized just once
  • a collection B of actors and joints which will be copied and deserialized multiple times

Collection B is incomplete, since it contains references to objects in A. When serializing B, the serialized format will remember each reference to an object in A using that object's ID (if it doesn't have an ID, then serialization will fail.) As long as an object of the right type with a matching ID is supplied when deserializing collection B, the reference can be resolved. Although collection B is incomplete, it is also said to be complete relative to collection A. For a formal definition of complete please refer to Complete.

../_images/Serialization_Dependency.png

Figure 2: Left: Collection A with Sharable Objects, Right: Collection B depending on A

Concretely, to serialize and deserialize an incomplete collection:

  • At serialization time, provide IDs for all objects in collection A that are referenced by objects in collection B.
  • When deserializing, provide a collection with matching IDs for all the objects in A that were referenced by objects in B.

Here are examples of how the application can provide identities (PxSerialObjectId) to express requirements of one collection to another. This can be done explicitly when adding the object with:

PxCollection* collection;
PxTriangleMesh* triMesh;
PxSerialObjectId triMeshId = 1;                                  // PX_SERIAL_OBJECT_ID_INVALID
                                                                 // is a reserved value

collection->add(*triMesh, triMeshId);

Or set the ID after adding the object:

collection->add(*triMesh);
collection->addId(*triMesh, triMeshId);

There is a helper function to generate IDs for all objects in a collection that do not have IDs yet:

PxSerialObjectId baseId = 1;                                     // PX_SERIAL_OBJECT_ID_INVALID is
                                                                 // a reserved value
PxSerialization::createSerialObjectIds(*collection, baseId);     // Assigns incremental ID values
                                                                 // to the collection objects

Already used ID values will be skipped by createSerialObjectIds, as well as objects that already have IDs.

After providing correct IDs, all required objects have been added to the collection to be serialized, but without adding the objects that are intended to be referenced. The complete function in PxSerialization supports completing a collection relative to another collection:

PxSerializationRegistry* registry;                               // Registry for serializable types

PxCollection* collectionB;                                       // Collection to be completed
PxCollection* collectionA;                                       // The collection, collectionB
                                                                 // will depend on

PxSerialization::complete(*collectionB, *registry, collectionA); // Completes collectionB, but
                                                                 // ignores objects in collectionA
                                                                 // (and also their requirements)

Serialization example:

PxConvexMesh** convexes;             // An array of mNbConvexes convexes
PxRigidDynamic** actors;             // An array of mNbConvexes actors referencing the convexes

PxSerializationRegistry* registry;   // Registry for serializable types
PxOutputStream& convexStream;        // Output stream for the convex collection
PxOutputStream& actorStream;         // Output stream for the actor collection

PxCollection* convexCollection = PxCreateCollection();
PxCollection* actorCollection = PxCreateCollection();

// Add convexes to collection
for(PxU32 i=0;i<mNbConvexes;i++)
    convexCollection->add(*convexes[i]);

// Create IDs for the convexes, starting with 1
PxSerialization::createSerialObjectIds(*convexCollection, PxSerialObjectId(1));

// Serialize the convexes along with their IDs

// Binary
   PxSerialization::serializeCollectionToBinary(convexStream, *convexCollection, *registry);
//~Binary

// RepX
   PxSerialization::serializeCollectionToXml(convexStream, *convexCollection, *registry);
//~RepX

// Add actors to other collection
for(PxU32 i=0;i<mNbActors;i++)
    actorCollection->add(*actors[i]);

// Add all required objects except the convexes
PxSerialization::complete(*actorCollection, *registry, convexCollection);

// Serialize the actors with references to convexCollection

// Binary
    PxSerialization::serializeCollectionToBinary(actorStream, *actorCollection, *registry,
                                                 convexCollection);
//~Binary

// RepX
    PxSerialization::serializeCollectionToXml(actorStream, *actorCollection, *registry, NULL,
                                              convexCollection);
//~RepX

// Release collections and registry
convexCollection->release();
actorCollection->release();
registry->release();

Deserialization example:

PxPhysics* physics;                         // The physics SDK object
PxSerializationRegistry* registry           // Registry for serializable types
PxCooking* cooking;                         // Cooking lib needed for instantiating objects (RepX)
PxScene* scene;                             // The scene into which the objects will be inserted

// Deserialize convexes along with their IDs (no external dependencies)

// Binary
    void* convexMemory128;                  // Aligned memory containing serialized convexes
    PxCollection* convexCollection =
        PxSerialization::createCollectionFromBinary(convexMemory128, *registry, NULL);
//~Binary

// RepX
    PxInputData& convexInputData = ...;     // Implemented by the application
    PxCollection* convexCollection =
        PxSerialization::createCollectionFromXml(convexInputData, *cooking, *registry, NULL);
//~RepX

// Deserialize actors referencing the convexCollection

// Binary
    void* actorMemory128;                   // Aligned memory containing serialized actors
    PxCollection* actorCollection =
        PxSerialization::createCollectionFromBinary(actorMemory128, *registry, convexCollection);
//~Binary

// RepX
    PxInputData& actorInputData = ...;      // Implemented by the application
    PxCollection* actorCollection =
        PxSerialization::createCollectionFromXml(actorInputData, *cooking, *registry,
                                                 convexCollection);
//~RepX

// Release convex collection
convexCollection->release();

// Add actors to scene and release collection and registry
scene->addCollection(*actorCollection);
actorCollection->release();
registry->release();

The next example shows how to deal with situations where the serialized objects require objects that are not serialized and deserialized but created by other means:

PxSerializationRegistry* registry;  // Registry for serializable types
PxMaterial** materials;             // Created procedurally by application
PxRigidDynamic** actors;            // An array of mNbConvexes actors referencing the convexes
PxOutputStream& actorStream;        // Output stream for the actor collection

// Add materials with IDs to collection
PxCollection* materialCollection = PxCreateCollection();

for(PxU32 i=0;i<mNbMaterials;i++)
    materialCollection->add(*materials[i], PxSerialObjectId(i+1));

// Create actor collection, complete and serialize
PxCollection* actorCollection = PxCreateCollection();

for(PxU32 i=0;i<mNbActors;i++)
    actorCollection->add(*actors[i]);

PxSerialization::complete(*actorCollection, *registry, materialCollection);

// Binary
    PxSerialization::serializeCollectionToBinary(actorStream, *actorCollection, *registry,
                                                 materialCollection);
//~Binary

// RepX
    PxSerialization::serializeCollectionToXml(actorStream, *actorCollection, *registry, NULL,
                                              materialCollection);
//~RepX

actorCollection->release();
materialCollection->release();          // Note that materialCollection was not serialized
registry->release();

Deserialization:

PxScene* scene;                         // The scene into which the objects will be inserted
PxSerializationRegistry* registry;      // Registry for serializable types
PxCooking* cooking;                     // Cooking library needed for instantiating objects(RepX)
PxMaterial** materials;                 // Created procedurally by application

// recreate material collection with consistent IDs, no deserialization
PxCollection* materialCollection = PxCreateCollection();

for(PxU32 i=0;i<mNbMaterials;i++)
    materialCollection->add(*materials[i], PxSerialObjectId(i+1));

// Deserialize actors with reference material collection

// Binary
    void* actorMemory128;                // aligned memory containing serialized actors
    PxCollection* actorCollection =
        PxSerialization::createCollectionFromBinary(actorMemory128, *registry,
        materialCollection);
//~Binary

// RepX
    PxInputData& actorInputData = ...;   // Implemented by the application
    PxCollection* actorCollection =
        PxSerialization::createCollectionFromXml(actorInputData, *cooking, *registry,
                                                 materialCollection);
//~RepX

materialCollection->release();
scene->addCollection(*actorCollection);
actorCollection->release();
registry->release();

Reference Counting of Deserialized Objects

This section assumes the background in Reference Counting.

Objects that are created by deserialization are always created with a reference that the application needs to give up by explicitly calling release(). The information whether the application gave up a reference to an object is not preserved on serialization.

See Shapes for a discussion of the method PxRigidActorExt::createExclusiveShape, which automatically releases the initial reference to the shape, leaving only the actor's reference. Again, the information that this reference has been released is not preserved by serialization.

Example for shapes:

PxOutputStream& outStream;          // Output stream for the collection
PxSerializationRegistry* registry;  // Registry for serializable types
PxRigidActor* actor;                // Any actor

// Creating shapes in different ways implies different rules for releasing

// Shape is automatically released when actor gets released
PxShape* shapeA = PxRigidActorExt::createExclusiveShape(*actor, ...);

// Shape is either created as "shared" or "exclusive" and needs to be released by
// the application
PxShape* shapeB = PxGetPhysics().createShape(...);
actor->attachShape(*shapeB);

// Create collection with actor and shapes and serialize
PxCollection* collection = PxCreateCollection();
collection->add(*actor);
collection->add(*shapeA);
collection->add(*shapeB);
PxSerialization::serializeCollectionToBinary(outStream, *collection, *registry, NULL);
collection->release();

// Releasing actors and shapes
actor->release();    // Releases actor and shapeA (automatically)
shapeB->release();   // Releases shapeB (necessary since shapeB was created through PxPhysics)

// Deserialize collection
...
void* memory128 = ...;   // Aligned memory for serialized data
collection = PxSerialization::createCollectionFromBinary(memory128, *registry, NULL);

// Release actors and release ALL shapes (necessary since shape creation history is
// not preserved across serialization
for(PxU32 i = 0; i < collection->getNbObjects(); i++)
{
    switch ( collection->getObject(i).getConcreteType() )
    {
        case PxConcreteType::eRIGID_DYNAMIC:
        case PxConcreteType::eRIGID_STATIC:
            static_cast<PxActor&>(collection->getObject(i)).release();   // Doesn't release
            break;                                                       // any shapes
        case PxConcreteType::eSHAPE:
            static_cast<PxShape&>(collection->getObject(i)).release();   // All shapes need to be
            break;                                                       // released explicitly
    }
}

Note

There is a PhysXExtensions function to release all objects within a collection: PxCollectionExt::releaseObjects.

Reconnecting PhysX and Application-Objects

Here is an example of how to fix up references with application objects by querying the IDs of a collection:

PxPhysics* physics;                     // The physics SDK object
PxCooking* cooking;                     // Cooking library needed for instantiating objects(RepX)
PxSerializationRegistry* registry;      // Registry for serializable types

// Deserialize objects along with IDs

// Binary
    void* memory128;                    // Aligned memory containing serialized objects
    PxCollection* collection =
        PxSerialization::createCollectionFromBinary(memory128, *registry, NULL);
//~Binary

// RepX
    PxInputData& inputData = ...;        // Implemented by the application
    PxCollection* collection =
        PxSerialization::createCollectionFromXml(actorInputData, *cooking, *registry,
                                                 materialCollection);
//~RepX

// Receive a list of all deserialized IDs
#define MAX_IDS 100
PxSerialObjectId idBuffer[MAX_IDS];
PxU32 numIds = collection->getIds(idBuffer, MAX_IDS);

// iterate over the list to patch up application objects
for (PxU32 i = 0; i < numIds; i++)
{
    PxActor* actor = collection->find(idBuffer[i])->is<PxActor>();
    if (actor)
    {
        // this assumes that findAppObjectFromId is able to locate
        // the corresponding application object from a PxSerialObjectId
        actor->userData = findAppObjectFromId(idBuffer[i]);
    }
}

Alternatively PxCollection::getObjects(...) and PxCollection::getId(PxBase& object) can be used to achieve the same.

Serializing Everything

PhysX provides two utility functions for serializing the entirety of the PhysX runtime: PxCollectionExt::createCollection(PxPhysics& sdk) and PxCollectionExt::createCollection(PxScene& scene):

PxPhysics* physics;                 // The physics SDK object
PxScene* scene;                     // The physics scene
PxSerializationRegistry* registry;  // Registry for serializable types
PxOutputStream& outStream;          // The user stream doing the actual write to disk

// 1) Create a collection from the set of all objects in the physics SDK that are shareable across
//    multiple scenes.
PxCollection* everythingCollection = PxCollectionExt::createCollection(*physics);

// 2) Create a collection from all objects in the scene and add it
//    to everythingCollection.
PxCollection* collectionScene = PxCollectionExt::createCollection(*scene);
everythingCollection->add(collectionScene);
collectionScene->release();

// 3) Complete collection
PxSerialization::complete(*everythingCollection, *registry);

// 4) serialize collection and release it

// Binary
    PxSerialization::serializeCollectionToBinary(outStream, *everythingCollection, *registry);
//~Binary

// RepX
    PxSerialization::serializeCollectionToXml(outStream, *everythingCollection, *registry);
//~RepX

everythingCollection->release();
registry->release();

Deserialization is as previously:

PxScene* scene;                      // The physics scene
PxCooking* cooking;                  // Cooking library needed for instantiating objects by RepX
PxSerializationRegistry* registry;   // Registry for serializable types

// Binary
    void* memory128 = ...;           // a 128-byte aligned buffer previously loaded from disk
                                     // by the user
    PxCollection* everythingCollection =
        PxSerialization::createCollectionFromBinary(memory128, *registry);
//~Binary

// RepX
    PxInputData& inputData = ...;    // Implemented by the application
    PxCollection* everythingCollection =
        PxSerialization::createCollectionFromXml(inputData, *cooking, *registry);
//~RepX

scene->addCollection(*everythingCollection);
everythingCollection->release();
registry->release();

Serializability

This section contains various definitions to describe serializability of a collection. Whether a collection can be successfully serialized and deserialized, optionally given an external references collection, can be queried by calling PxSerialization::isSerializable(...)

Requires

An object A requires another object B if A maintains a reference to B that needs to be re-established for successfully deserializing A. This implies that B needs to be deserialized before A.

Here is the table of the relationship requires of all PhysX objects:

joints require their actors and constraint
rigid actors require their shapes
shapes require their materials and mesh (triangle mesh, convex mesh or height field), if any
articulations require their links and joints
aggregates require their actors

Subordinate

Subordinates are objects that cannot be instantiated without being owned by other objects. An articulation link, for example, can only be instantiated as part of its articulation.

The following three types are subordinates:

articulation links
articulation joint
constraints

Complete

Definition of a complete set:

A set of objects C is complete if every object required by C is in C.

Definition of a set that is complete relative to another set:

A set of objects C is complete relative to a set D if every object required by C is in C or in D. This means that C can be deserialized given D.

Serializable

Here is the complete set of requirements on a collection C with dependencies to D such that C can be serialized:

  • C is complete relative to D. ("no dangling references")
  • Every object in D required by an object in C has a valid ID. ("no unnamed references")
  • Every subordinate object in C is required by another object in C. ("no orphans")

Binary Serialization Specifics

The following sections describe specific properties of the binary serialization system.

Memory Management

Management of memory blocks containing deserialized objects is left to users. It is the user's responsibility to:

  • allocate the memory block. Note that it must be properly aligned, to a PX_SERIAL_FILE_ALIGN (128) bytes boundary.
  • fill the block with serialized data, typically by loading it from disk.
  • deallocate the memory block when the objects within have been released by PhysX.

Although the user owns the memory block, the PhysX runtime owns any deserialized objects it contains. Concretely, calling release() on an object that was created by deserialization will cause its destructor to run, but will not deallocate its memory. If the block is deallocated before the destructors have run for all the objects it contains, the PhysX runtime will likely crash. For more information about how deserialized objects need to be released see Reference Counting of Deserialized Objects.

Versioning

The binary serialized data is typically specific to the version of the SDK it was produced with. However, a SDK version can load the data of older SDK versions if the binary format didn't change. This is usually the case with bugfix releases. The compatible SDK versions are listed in the code documentation of PX_BINARY_SERIAL_VERSION in PxSerialization.h.

Retargeting to other Platforms

Binary serialized data is platform-specific, and when serialized it always targets the platform on which it was created. The binary converter in the extensions library retargets data from one platform to another. Typically assets are serialized on an authoring platform (Windows, Mac OS X and Linux). The serialized data can then be retargeted, for example, to a console or any other runtime platform.

The converter requires meta-data for the source and target platforms, which contains information about the binary layout of objects for that platform. To obtain metadata, use the function provided in the extensions library for each platform:

void PxSerialization::dumpBinaryMetaData(PxOutputStream& stream, PxSerializationRegistry& sr);

On each target platform, run it once and keep generated data around. Alternatively a set of pre-built binary metadata is included with the PhysX SDK at [path to installed PhysX SDK]/Tools/BinaryMetaData.

../_images/Serialization_Retargeting.png

Figure 3: Schema of Retargeting

Assuming that the extensions library has been initialized, conversion takes place as follows:

PxSerializationRegistry* registry;      // Registry for serializable types
PxInputStream& srcMetadata;             // metadata for the 'from' platform
                                        // (e.g. PxDefaultFileInputData)
PxInputStream& dstMetadata;             // metadata for the 'to' platform

PxInputStream& srcAsset;                // stream containing source asset
PxU32 srcAssetSize;                     // size of the source asset
PxOutputStream& dstAsset;               // output stream for retargeted asset

PxBinaryConverter* converter = PxSerialization::createBinaryConverter();
converter->setMetaData(srcMetadata, dstMetadata);
converter->convert(srcAsset, srcAssetSize, dstAsset);

The Convert Tool

The convert tool is at [path to installed PhysX SDK]/Snippets/SnippetConvert. It illustrates how to convert PhysX 3 serialized binary files from one platform to another. It only compiles and runs on authoring platforms (Windows, MacOs and Linux).

SnippetConvert is a simple command-line tool supporting the following options:

--srcMetadata=<filename>            Defines source metadata file
--dstMetadata=<filename>            Defines target metadata file
--srcBinFile=<filename>             Source binary file to convert
--dstBinFile=<filename>             Outputs target binary file
--generateExampleFile=<filename>    Generates an example file
--verbose                           Enables verbose mode

Object Names

Some SDK objects, such as shapes and actors, can be given names using the PxShape::setName() and PxActor::setName() functions. By default these names are not serialized. The 'exportNames' parameter of the PxSerialization::serializeCollectionToBinary() can be set to true in order to serialize the names along with the objects.

API-level Serialization (RepX) Specifics

RepX stands for Representation X and is the ASCII-XML serialization format for PhysX 3. As opposed to binary serialization, the RepX XML serialization is not intended to be used in performance critical or memory constrained situations. The following sections describe specifics of the RepX XML serialization system.

Upgrading RepX Data

Upgrading RepX data from an older PhysX version to a newer one is easy. It happens implicitly when deserializing old RepX data with a newer PhysX SDK and re-serializing the resulting PxCollection.

Example for upgrading a RepX stream:

PxPhysics* physics;                                       // The physics SDK object (e.g.
                                                          // PhxsX 3.3)
PxCooking* cooking;                                       // Cooking library needed for
                                                          // instantiating objects
PxSerializationRegistry* registry;                        // Registry for serializable types

PxDefaultFileInputData inputData(pathTo30RepXFile);       //load an older 3.x RepX file
PxCollection* collection =
    PxSerialization::createCollectionFromXml(inputData, *cooking, *registry);

PxDefaultFileOutputStream outStream(pathToNewRepXFile);
PxSerialization::serializeCollectionToXml(outStream, *collection, *registry);

Object Names

As opposed to binary serialization, the object names that can be specified with the PxShape::setName() and PxActor::setName() functions, are always included in the serialized format. On deserialization with PxSerialization::createCollectionFromXml(...) the names can be recovered by setting the PxStringTable parameter.

If PxStringTable parameter is set, the names will live within the memory which is allocated by the string table. The string table must not be released unless it can be guaranteed that the names will not be accessed any more.

Caching Cooked Geometry Data

In order to facilitate faster instantiation of XML data, it is possible to configure the XML serialization to store the cooked triangle and convex mesh data along with the plain data. The cooked data caching can be enabled by passing a PxCooking instance into PxSerialization::serializeCollectionToXml(...). The cached cooked data is ignored when its format is incompatible with the current SDK version.

Common Use Cases

API-level RepX serialization should be used whenever compatibility and human readability are important. The PhysX plug-ins for the DCC tools 3ds Max and Maya use RepX to export PhysX objects. The resulting RepX files can then be deserialized and loaded into the PhysX runtime. This is useful for rapid prototyping or for generally loading PhysX assets if performance is not of a big concern. For quick loading of assets it is better to convert RepX data into binary serialized data. RepX is also useful for reproducing situations with unwanted behavior without the need to provide the whole application. For this, the application may be connected to the PhysX Visual Debugger (PVD), which records the scene of interest. A representative frame can then be saved in RepX format from within PVD (see PVD).

Binary serialization should be used in performance and memory constrained situations. The main target use-case is streaming in chunks of a large game level that can't be loaded into memory at once. Creating and loading save games is another application that could be optimized by using binary serialization. PhysX objects in binary format can also be sent over the network to enable efficient game state synchronization.

Snippet Discussion

The following snippets illustrate common operations such as managing collections, serialization, deserialization and re-targeting of binary data.

SnippetSerialization

SnippetSerialization shows binary and XML serialization of a scene with a number of jointed rigid bodies representing a chain. This is done in a way that allows the instantiation of multiple chains while sharing the shape and the material across all chains. The snippet shows how to create and populate collections, specify IDs to enable resolving dependencies, serialize collections, deserialize collections and add actors to the scene for simulation.

The snippet also shows how to allocate a data block aligned to 128 bytes and demonstrates how to copy binary serialized data into it. It further demonstrates that the data blocks containing the binary deserialized collections must be maintained until the corresponding objects are not needed anymore and have been released.

../_images/Serialization_Snippet.png

Figure 4: SnippetSerialization

SnippetConvert

SnippetConvert illustrates how binary serialized data can be re-targeted from an authoring platform to a runtime platform such as a console. The snippet is a simple command line tool that can load a binary serialized data file along with meta data files for both source and destination platforms and then output a converted binary data file. See the snippet's source documentation for more details on usage.

SnippetLoadCollection

SnippetLoadCollection shows how to deserialize serialized collections from either binary or XML format. The snippet is a command line tool that can connect to the PhysX Visual Debugger application and display the content of serialized collection files. See the snippet's source documentation for more details.

Best practices / Troubleshooting

  • Concurrent simulation and serialization is not supported and leads to undefined behavior.
  • If releasing PhysX objects leads to crashes or errors it is possible that the application is releasing some objects twice. The following two reasons should be considered: 1.) A potential source of error is to release PhysX objects without updating collections referencing these objects. 2.) Shapes that where created through an actor have their application reference automatically released on creation. If such a shape is serialized and deserialized the creation history will be lost. It might be convenient to use the extension function PxCollectionExt::releaseObjects because it deals with the different cases as required. See Reference Counting of Deserialized Objects.
  • If accessing binary deserialized PhysX objects, including accesses during simulation, causes crashes it might be due to the premature release of the memory block that holds the deserialized objects.
  • If binary files are too large and/or too slow to load it might be that shared assets have been serialized multiple times. An example of a shared asset might be a mesh that is referenced by multiple shapes. The solution is to separate shared PhysX objects into a separate collection. See Serializing Incomplete Collections.
  • If loading PhysX objects from RepX files is too slow two things should be considered: 1.) Could binary serialization be used instead? Even for debugging it might make sense to convert RepX files into binary serialized data by re-serializing them with the binary approach. 2.) Meshes tend to load very slowly from text files. RepX serialization offers an option to cache cooked mesh data by in-lining binary data into the RepX file. If such a cache is present and valid, the loading can become significantly faster. See Caching Cooked Geometry Data.

PVD

The PhysX Remote Debugger provides the functionality to export single frames of PhysX scenes as RepX files. The resulting files can be used to playback a snapshot of the PhysX state. In many cases this is sufficient to isolate an issue. The option can be found in the menu of PVD: [Menu > File > Export Current Frame To RepX]

../_images/Serialization_PVD.png

Figure 5: RepX Functionality in PVD