Skip to content
Back to top
View this page

# Serialization

Viper provides multiple serialization formats for values and definitions. This chapter covers JSON and binary encoding.

## JSON Encoding

### Encoding Values

Encode any value to JSON:

    >>> from dsviper import *

    >>> v = Value.deduce([1, 2, 3])
    >>> Value.json_encode(v)
    '[1,2,3]'

Complex values — Python tuples become JSON arrays:

    >>> v = Value.deduce({(1, 2): "one", (3, 4): "two"})
    >>> Value.json_encode(v)
    '[[[1,2],"one"],[[3,4],"two"]]'

### Decoding Values

Decode JSON back to a value:

    >>> json_str = '[1,2,3]'
    >>> t = TypeVector(Type.INT64)
    >>> v = Value.json_decode(json_str, t, Definitions().const())
    >>> v
    [1, 2, 3]

### Pretty Printing

Define a small `Login` structure to demonstrate pretty printing:

    >>> defs = Definitions()
    >>> ns = NameSpace(ValueUUId("f529bc42-0618-4f54-a3fb-d55f95c5ad03"), "Tuto")
    >>> desc = TypeStructureDescriptor("Login")
    >>> desc.add_field("nickname", Type.STRING)
    >>> desc.add_field("password", Type.STRING)
    >>> t_login = defs.create_structure(ns, desc)

Format JSON with indentation:

    >>> v = Value.create(t_login, {"nickname": "alice", "password": "secret"})
    >>> print(Value.json_encode(v, indent=2))
    {
      "nickname": "alice",
      "password": "secret"
    }

## DSM Definitions Serialization

### To JSON

Convert `Definitions` to a `DSMDefinitions` snapshot, then encode to JSON:

    >>> defs2 = Definitions()
    >>> ns2 = NameSpace(ValueUUId("aaaaaaaa-0000-0000-0000-000000000001"), "Demo")
    >>> defs2.create_concept(ns2, "User")
    Demo::User

    >>> dsm_defs = defs2.const().to_dsm_definitions()
    >>> json_str = dsm_defs.json_encode(indent=2)
    >>> "Demo" in json_str and "User" in json_str
    True

The encoded JSON contains the standard DSM categories — namespaces, concepts, structures, enumerations, attachments, clubs, function pools, and attachment function pools.

### From JSON

    >>> restored = DSMDefinitions.json_decode(json_str)
    >>> type(restored).__name__
    'DSMDefinitions'

### To DSM Language

Render definitions back to DSM source:

    >>> print(dsm_defs.to_dsm())
    namespace Demo {aaaaaaaa-0000-0000-0000-000000000001} {

    concept User;

    }; // ns Demo

## Binary Encoding

Binary encoding is compact and fast, used for persistence and network transfer.

### Encoding Values

`Value.encode(value)` returns a `ValueBlob`; `Value.decode(blob, type, defs)` restores the value:

    >>> v = Value.deduce([1, 2, 3])
    >>> blob = Value.encode(v)
    >>> blob
    blob(...)

    >>> restored = Value.decode(blob, v.type(), Definitions().const())
    >>> restored == v
    True

### Encoding Definitions

`Definitions` are encoded via the const view:

    >>> blob = defs2.const().encode()
    >>> blob
    blob(...)

    >>> restored = Definitions.decode(blob)
    >>> type(restored).__name__
    'Definitions'

## Stream Codec

For custom binary formats and RPC, Viper provides low-level stream codecs that encode/decode primitive types directly.

### Available Codecs

Codec | Description
--- | ---
`Codec.STREAM_BINARY` | Compact binary, no type checking
`Codec.STREAM_TOKEN_BINARY` | Adds type tokens for validation

The token variant detects desynchronization early by prefixing each value with its type.

### Encoding

Create an encoder, write primitives, finalize:

    >>> encoder = Codec.STREAM_BINARY.create_encoder()
    >>> encoder.write_int64(42)
    >>> encoder.write_float(3.14)
    >>> encoder.write_string("hello")

    >>> blob = encoder.end_encoding()
    >>> blob
    blob(...)

### Decoding

Read in the same order. Note that `write_float` / `read_float` round-trip through 32-bit float, so `3.14` comes back with float32 precision:

    >>> decoder = Codec.STREAM_BINARY.create_decoder(blob)
    >>> decoder.read_int64()
    42
    >>> decoder.read_float()
    3.140000104904175
    >>> decoder.read_string()
    'hello'

    >>> decoder.has_more()
    False

### Type Safety with Tokens

The token codec rejects mismatched reads:

    >>> encoder = Codec.STREAM_TOKEN_BINARY.create_encoder()
    >>> encoder.write_int64(42)
    >>> blob = encoder.end_encoding()

    >>> decoder = Codec.STREAM_TOKEN_BINARY.create_decoder(blob)
    >>> decoder.read_bool()
    Traceback (most recent call last):
        ...
    dsviper.ViperError: ...Expected token bool, got int64...

### Computing Sizes

    >>> sizer = Codec.STREAM_BINARY.create_sizer()
    >>> sizer.size_of_float()
    4
    >>> sizer.size_of_int64()
    8

## Value Description

Get a descriptive string representation:

    >>> v = Value.deduce([1, 2, 3])
    >>> v.description()
    '[1, 2, 3]:vector<int64>'

    >>> login = Value.create(t_login, {"nickname": "alice"})
    >>> login.description()
    "{nickname='alice':string, password='':string}:Tuto::Login"

## Binary File Format

DSM definitions can be saved in binary format (`.dsmb`):

    # Write definitions to binary file
    >>> with open("model.dsmb", "wb") as f:
    ...     f.write(defs.const().encode())

    # Read definitions from binary file
    >>> with open("model.dsmb", "rb") as f:
    ...     defs = Definitions.decode(f.read())

## Common Patterns

### Round-Trip Test

Verify encoding/decoding preserves data:

    >>> original = Value.create(t_login, {"nickname": "test"})
    >>> json_str = Value.json_encode(original)
    >>> decoded = Value.json_decode(json_str, t_login, defs.const())
    >>> original == decoded
    True

### Schema Export

Export schema for external systems:

    >>> defs = db.definitions()
    >>> dsm_defs = defs.to_dsm_definitions()
    >>> print(dsm_defs.to_dsm())          # DSM language
    >>> print(dsm_defs.json_encode(indent=2))  # JSON schema

Note

The `Schema Export` snippet above pulls definitions from a live `db` and therefore requires a working database. See Database for a full walkthrough.

## Summary

Format | Use Case | API
--- | --- | ---
JSON | Debugging, interop | `Value.json_encode/decode`
Binary | Persistence, RPC | `Value.encode/decode`
DSM | Human-readable schema | `dsm_defs.to_dsm()`

## What’s Next

- DSM Processing - Loading and introspecting DSM files

- DSM Manual - Data modeling language

- Toolchain - Development tools