Skip to content
Back to top
View this page

# Binary Data (Blobs)

Blobs provide efficient binary data storage with typed layouts and zero-copy NumPy integration.

When to use: Use blobs for binary data like images, meshes, and raw buffers. `BlobArray` for typed arrays, `BlobPack` for structured multi-region data.

## Quick Start

    from dsviper import BlobLayout, BlobArray, BlobPack, BlobPackDescriptor
    import numpy as np

    # Typed array: 100 vec3 positions (float, 3 components)
    layout = BlobLayout('float', 3)
    positions = BlobArray(layout, 100)

    # Zero-copy NumPy access
    np_view = np.array(positions, copy=False)
    np_view[0] = [1.0, 2.0, 3.0]  # Modifies original

    # Structured blob: mesh with multiple regions
    desc = BlobPackDescriptor()
    desc.add_region('positions', BlobLayout('float', 3), 100)
    desc.add_region('normals', BlobLayout('float', 3), 100)
    desc.add_region('indices', BlobLayout('uint', 3), 50)
    mesh = BlobPack(desc)

    # Access region as NumPy array
    pos_np = np.array(mesh['positions'], copy=False)

## Choosing the Right Class

Data Type | Class | Example
--- | --- | ---
Raw bytes | `ValueBlob` | `ValueBlob(bytes_data)`
Typed array | `BlobArray` | `BlobArray(layout, count)`
Multiple regions | `BlobPack` | Mesh with pos/normals/indices
Database reference | `ValueBlobId` | SHA-1 hash reference
Large files (>2GB) | `BlobStream` | Streaming upload

## Core Classes

`dsviper.BlobLayout` | A class used to describe the layout of the element in a blob.
`dsviper.BlobArray` | A class used to reinterpret the blob of the internal BlobView as an array<blob_layout.data_type>.
`dsviper.BlobPack` | A class used to implements memory regions in one blob.
`dsviper.BlobPackDescriptor` | A class used to describe binary regions.
`dsviper.BlobPackRegion` | A class used for a region that implement the buffer protocol.

## Blob I/O

`dsviper.BlobStream` | A class used to copy a huge blob (> 2GB) to a database.
`dsviper.BlobEncoder` | A class used to encode a blob.
`dsviper.BlobEncoderLayout` | A class used to represent the layout of one element.
`dsviper.BlobView` | A class used to interpret the bytes of a blob from the layout.
`dsviper.BlobData` | A class used to represent the data associated with a blob during the synchronization of two databases.

## Database Integration

`dsviper.BlobGetting` | An interface used to retreive blobs from a persistence layer.
`dsviper.BlobInfo` | A class used to represent various information of a blob in the persistence layer.
`dsviper.BlobStatistics` | A class used to represent the statistics about blobs.