Data modeling & encoding

Separate engine and application schemas

ZeeDB persists bytes. Your application defines what those bytes mean and how records relate to one another. Model these decisions explicitly:

Prefer collection constants in your application so a typo does not silently create a separate namespace. There is no collection-creation DDL or schema registry.

Collection namespaces

Collection names are nonempty strings containing at most 255 bytes and no NUL. Names are compared as supplied; ZeeDB does not normalize case or Unicode.

Collections are not directories on disk and do not confer permissions. For independent database ownership, use different dedicated directories. Apply operating-system access control outside the engine.

Lexicographic byte order

String keys sort by bytes:

"1", "10", "100", "2", "20"

For fixed-width unsigned integers, big-endian encoding preserves ascending numeric order:

func encodeID(id uint64) []byte {
    key := make([]byte, 8)
    binary.BigEndian.PutUint64(key, id)
    return key
}

Import encoding/binary. This unsigned recipe does not define signed-integer, floating-point, timestamp, or descending ordering; choose and test those encodings separately.

Composite keys and prefix scans

A string key such as account/42/event/000123 can group related records. Establish an escaping/length-prefix policy if user-supplied components can contain separators. Simple concatenation without an unambiguous encoding can collide.

Prefix scans use a start equal to the prefix and an exclusive upper bound found by incrementing its rightmost non-0xff byte and truncating after it. Appending 0xff is not a general prefix upper bound. See the pagination recipe.

Keep key encodings stable. Changing numeric width, casing, separators, or canonicalization changes identity and sort order. Treat that as a data migration.

JSON values and versioned envelopes

{
  "schema": 1,
  "user": { "id": "1024", "name": "Ada" }
}

Use a schema field inside the value. It is independent of Record.Version, which records the database commit sequence.

Validate application fields before writing. Validate the schema and any key/payload relationship after decoding. An engine checksum can detect accidental storage damage; it does not prove that JSON is valid or the record is logically consistent.

Choose how unknown fields, missing fields, and newer schemas are handled. A permissive JSON decoder and a strict decoder have different compatibility behavior. The JSON store recipe demonstrates a small versioned envelope.

Empty value versus missing record

An empty []byte{} value is valid. Its length does not indicate absence. Only a returned ErrNotFound means that the record is missing/deleted.

value, err := db.Get(ctx, "settings", []byte("optional"))
if errors.Is(err, zeedb.ErrNotFound) {
    // Supply the application default.
} else if err != nil {
    return err
} else {
    // value may legitimately have length zero.
    _ = value
}

TTL, indexes, and counters

No TTL expiry, query engine, automatic index maintenance, or atomic increment is built in. An expiration timestamp in JSON is only metadata until your application checks it. A manually maintained index must be written with primary data in the same batch and coordinated for concurrent updates.

Avoid uncoordinated Get → increment → Put counters. Serialized commits do not serialize the earlier read. See concurrency and manual indexes.

Schema changes

For a small migration, stop writers, make a closed-directory backup, open with the original options, read/decode old values, and write validated replacements in bounded batches. A whole multi-batch migration is not atomic. Make progress and retry behavior explicit at the application layer.

Do not change the engine’s internal WAL/table/manifest bytes to migrate application JSON. File-format migration and application schema migration are separate responsibilities. Neither has built-in migration tooling today.