Serializing and deserializing data
This document describes how data in the C3 Agentic AI Platform is serialized and deserialized over a network and also explains the various serialization formats supported by the platform.
Default JSON serialization
JSON is the standard serialization format for C3 AI values. There are two variants of JSON, mapped to the JsonType Type: untyped and typed.
untypedJSON represents the raw serialization of C3 AI values without any Type information. Its DSL name is simplyjson.typedJSON is used for full-fidelity communication between type-system-aware processes or storage. Its DSL name istyped json.
For the following examples, assume the following Type definitions:
type Animal mixes Named
type Dog mixes Animal
type Person {
pet: Animal
bestFriend: Dog
}Untyped JSON serialization
untyped JSON is a compact representation of the data contained in C3 AI values without any Type information. The resulting JSON string includes all the keys and values from the original data structure, without any additional metadata or formatting.
The following table shows how values are serialized.
| Source Type | Source Value | Serialization |
|---|---|---|
| int | 42 | 42 |
| string | "foo" | "foo" |
| [int] | [1, 2, 3] | [1, 2, 3] |
| map | {"foo": 42} | {"foo": 42} |
| Dog | Dog({name: "Fido"}) | {name: "Fido"} |
| Person | Person({pet: Dog({name: "Fido"}), bestFriend: Dog({name: "Snoopy"})}) | {pet: {name: "Fido"}, bestFriend: {name: "Snoopy"}} |
| any | "foo" | "foo" |
| [any] | [Dog({name: "Fido"}), 42] | [{name: "Fido"}, 42] |
Typed JSON serialization
typed JSON contains additional Type information and the raw data of untyped JSON so that it can reliably deserialize the JSON and reproduce the original value in a Type-aware process. The implementation of typed JSON aims to ensure efficiency and compactness while still containing all the necessary Type information.
An in-memory value produces typed JSON. The typed JSON depends on the target ValueType being serialized. The target Type refers to the declared ValueType of the field or method parameter utilizing the value.
- If the Type of the in-memory instance exactly matches the target Type, then no additional Type information must be included in the JSON.
- Otherwise, when the Type of the in-memory instance is a subtype of the target Type, or the target Type is
anyoranyof, then Type information must be included so that the deserializing process can reliably reproduce the original value as the correct Type.
When Type information must be included in the JSON representation of a value, then:
- If the value is an instance of a
Typethat mixes WithType, the produced JSON should havetypeas its first key and be the same as theuntypedrepresentation of the value. - Otherwise, we use the parametric Boxed Type, which has fields
typeandvalue, to Boxed#box and serialize the value's data and Type. Boxed mixes Obj (and, transitively, WithType) and can therefore follow the standard serialization of WithType. By convention, however, we omitBoxed<>in the serialization ofBoxedvalues. For example, a boxedinthas TypeBoxed<int>, but would normally be serialized as{'type': "int", value: 42}instead of{'type': "Boxed<int>", value: 42}. - Because WithType instances have a natural serialization that includes their Type information, they should never be explicitly boxed. For example,
{'type': "Dog", 'value': {'name': "Fido"}}is incorrect and{'type': "Dog", 'name': "Fido"}is correct.
Note: In the following table, a plain anyof refers to any anyof combination (DSL type1 | type2 | ...)
| Source Type | Source Value | Target Type | Serialization |
|---|---|---|---|
| int | 42 | int | 42 |
| int | 42 | any | {type: "int", value: 42} |
| string | "foo" | string | "foo" |
| string | "foo" | string | int |
| [int] | [1, 2, 3] | [int] | [1, 2, 3] |
| map | {"foo": 42} | map | {"foo": {type: "int", value: 42}} |
| Dog | Dog({name: "Fido"}) | Dog | {name: "Fido"} |
| Dog | Dog({name: "Fido"}) | Animal or any or anyof | {type: "Dog", name: "Fido"} |
| Person | Person({pet: Dog({name: "Fido"}), bestFriend: Dog({name: "Snoopy"})}) | Person | {pet: {type: "Dog", name: "Fido"}, bestFriend: {name: "Snoopy"}} |
| Person | Person({pet: Dog({name: "Fido"}), bestFriend: Dog({name: "Snoopy"})}) | any | {type: "Person", pet: {type: "Dog", name: "Fido"}, bestFriend: {name: "Snoopy"}} |
| [Dog] | [Dog({name: "Fido"}), Dog({name: "Snoopy"})] | [Dog] | [{type: "Dog", name: "Fido"}, {type: "Dog", name: "Snoopy"}] |
| [Dog] | [Dog({name: "Fido"}), Dog({name: "Snoopy"})] | [Dog] | [Person] |
| [any] | [Dog({name: "Fido"}), 42] | [any] | [{type: "Dog", name: "Fido"}, {type: "int", value: 42}] |
| [any] | [Dog({name: "Fido"}), 42] | any | {type: "[any]", value: [{type: "Dog", name: "Fido"}, {type: "int", value: 42}]} |
Deserialization of typed JSON
All language SDKs should deserialize typed JSON to produce correctly Typed instances of the original (serialized) values. User code should never have to deal with Boxed instances even if boxing is required to preserve Type information during serialization.
/**
* Example implementation of ValueType.valueFromJson
* @param json a potentially "boxed" json
*/
function valueFromJson(json) {
if (typeof json === 'object' && typeof json.type === 'string') {
var typ = Type.fromString(json.type);
// We almost always want to unbox, except for the following scenario:
// `json` contains "Boxed<int>" and `this` is also "Boxed<int>". In that case, the caller should get back
// an instance of Boxed<int>.
if (typ.meta().isA(Boxed) && (this.isAny() || !typ.meta().referenceType().isA(this))) { // `this` is the current ValueType.
var binding = typ.meta().varBinding('V', true);
if (!binding.isA(this)) {
// Check to make sure the ValueType in the box isA expected ValueType. Ideally only running in CI.
// This check cannot be enabled until we figure out the [int] vs Array<int> vs DataTable issue.
throw new Error(`ValueType mismatch during unboxing: "${binding}" is not a "${this}".`);
}
var desered = typ.fromJson(json); // Boxed isA Obj, therefore isA Serializable.
return desered.value;
}
}
// The rest of the "normal" deser logic.
// E.g. BinaryType should convert json to binary, ArrayType should turn json into instance of Array type...
}Deserialization of json from non type system aware clients
The C3 Agentic AI Platform does its best to guess the value Type of the value passed in and uses the rules below to deserialize values from a non-type-system-aware client.
| From the wire | Target Type (any) | Target Type (Obj) | Target Type (anyof)* |
|---|---|---|---|
| [1,2,3] | [1,2,3] | {type: "[int]", value: [1,2,3]} | [1,2,3] |
| {a: 1, b: 2} | {a: 1, b: 2} | {type: "map<string, int>", value: {a: 1, b: 2}} | {a: 1, b: 2} |
| {"id": "foo"} | {"id": "foo"} | {type: "map<string, string>", value: {"id": "foo"}} | {type: "User", id: "foo"} |
| *anyof(int, string, [int], [string], map<string,int>, User) |
Serializable
Mix this Type if your Type needs custom serialization. Obj already mixes Serializable so all Obj's are automatically serialized in all AI supported runtimes. Unless your Type specifically mixes Value you do not need to do anything specific to make instances of your Type be serializable.
If your Type explicitly mixes Value, you must implement all abstract methods on Serializable#toJson, Serializable#toTypedJson, Serializable#fromJson for C3 AI to be able to serialize and deserialize instances of your Type - in all languages (including Java, JavaScript, and Python) from where instances of these objects must be serialized and deserialized.
type SampleType mixes Value, Serializable {
customField: native
toJson: ~
toTypedJson: ~
fromJson: ~
}If your Type is not a Value but you want to implement custom serialization and deserialization for your Type, you can mix Serializable and provide a specific implementation for it.
type SampleType mixes Serializable {
field1: string
field2: int
toJson: ~
toTypedJson: ~
fromJson: ~
}You can choose to provide an implementation in a specific runtime, and the sdk uses the overrides only in those specific runtimes.
StringSerializable
If a Type mixes StringSerializable but not Serializable, for example:
type StringBear mixes StringSerializable, Valuethen it should be serialized and deserialized using StringSerializable#toString and StringSerializable#fromString. Both methods should be implemented in all languages in which this Type is expected to work.
If a Type mixes both Serializable and StringSerializable, the various serialization and deserialization methods used on strings, deserialize, and XML must use the same semantics. For example, each of the to*/from* method pairs should be able to fully recreate any Bear object.
Consider the code snippet below. The custom Type Bear mixes Serializable and StringSerializable.
type Bear mixes Serializable, StringSerializableThen the following poly test is true for all instances of Bear:
bear = Bear.make(...)
b1 = Bear.fromString(bear.toString())
b1.isSame(bear) === true
b2 = Bear.fromJson(bear.toJson())
b2.isSame(bear) === true
b3 = Bear.fromJsonString(bear.toJsonString())
b3.isSame(bear) === true
b4 = Bear.fromXmlString(bear.toXmlString())
b4.isSame(bear) === trueSpecial cases
There are special cases when renaming a field to the string value "type". Please refer to the documentation on Ann.Ser.
The field name type in the input data is used to identify which Type to use for deserialization. However, it is possible that another name may be used, such as when the name of the type() method of a Type that mixin WithType has been renamed with @ser, in which case the later name should be used.
Note: that if another field is annotated to use the serialized name of the type method, the base Type can be used to identify which Type to use for deserializing purposes.