Worker lifecycle & shutdown

Run the example

go run ./examples/lifecycle

reopened: 20 records from 4 concurrent workers

The complete source below comes directly from examples/lifecycle/main.go. It uses one temporary directory and one shared handle, not one handle per worker.

Own the handle at application scope

Open once after configuration validation. Pass the handle into workers/repositories without transferring responsibility for closing it. The application owner closes only after all users finish.

The example launches four workers with five distinct keys each. Distinct keys avoid claiming that a concurrent read-modify-write invariant is protected by engine commit serialization.

Bound workers and error delivery

const workers = 4
failures := make(chan error, workers)
var wg sync.WaitGroup

Each worker reports at most one error. A channel sized to the worker count allows errors to be reported while the parent waits. Each goroutine calls wg.Done() on every path.

On a write failure the worker cancels the shared producer context. Cancellation is cooperative and does not roll back commits already started by other workers.

Drain before close

wg.Wait()
close(failures)
for failure := range failures {
    err = errors.Join(err, failure)
}

Only the parent closes the error channel after every producer exits. The example refuses to continue to its success path if any worker failed.

In a service, stop admitting requests before this draining step. Do not let background jobs use a handle after the owning service closes it.

Checkpoint and reopen

The example checkpoints after workers finish, explicitly closes the DB, reopens the same directory, and verifies the logical record count.

Flush is optional during normal shutdown; WAL sync is the durability boundary. It is included here to exercise a checkpoint/reopen path. Close errors remain significant even if writes returned nil earlier.

Preserve shutdown errors

The runner uses named return errors and errors.Join so file close/cleanup failures do not disappear. It also demonstrates that a deferred second Close is harmless because the method is idempotent.

For your application’s real data, do not remove its directory on shutdown. Automatic removal in these examples applies only to the directories they create with os.MkdirTemp.

Complete source

This listing is embedded from the runnable example at build time, including its bounded workers, error delivery, shutdown, checkpoint, and reopen verification.

// Run from the repository root: go run ./examples/lifecycle
package main

import (
	"context"
	"errors"
	"fmt"
	"os"
	"sync"

	zeedb "github.com/Berlindraft/ZeeDB"
)

func main() {
	if err := run(); err != nil {
		fmt.Fprintln(os.Stderr, err)
		os.Exit(1)
	}
}
func run() (err error) {
	dir, err := os.MkdirTemp("", "zeedb-lifecycle-")
	if err != nil {
		return err
	}
	defer func() { err = errors.Join(err, os.RemoveAll(dir)) }()
	db, err := zeedb.Open(dir, zeedb.Options{})
	if err != nil {
		return err
	}
	defer func() { err = errors.Join(err, db.Close()) }()
	ctx, cancel := context.WithCancel(context.Background())
	defer cancel()
	const workers = 4
	failures := make(chan error, workers)
	var wg sync.WaitGroup
	for worker := range workers {
		wg.Add(1)
		go func() {
			defer wg.Done()
			for n := range 5 {
				key := []byte(fmt.Sprintf("worker-%d/item-%d", worker, n))
				if err := db.Put(ctx, "jobs", key, []byte("done")); err != nil {
					failures <- err
					cancel()
					return
				}
			}
		}()
	}
	// Drain producers before closing: never let background work outlive the DB.
	wg.Wait()
	close(failures)
	for failure := range failures {
		err = errors.Join(err, failure)
	}
	if err != nil {
		return err
	}
	if err := db.Flush(context.Background()); err != nil {
		return err
	}
	if err := db.Close(); err != nil {
		return err
	}
	// Close is idempotent, including the deferred Close above.
	reopened, err := zeedb.Open(dir, zeedb.Options{})
	if err != nil {
		return err
	}
	defer func() { err = errors.Join(err, reopened.Close()) }()
	stats, err := reopened.Stats()
	if err != nil {
		return err
	}
	if stats.Records != workers*5 {
		return fmt.Errorf("expected 20 records, got %d", stats.Records)
	}
	fmt.Printf("reopened: %d records from %d concurrent workers\n", stats.Records, workers)
	return nil
}