MoonKV

MoonKV is a small embedded, append-only key-value store written entirely in MoonBit. It follows the Bitcask model: writes are appended to a write-ahead log, point reads use an in-memory key directory, and compaction produces a data segment plus a hint file for faster restart recovery.

What is included

• Append-only records with FNV-1a checksums and tombstones.
• In-memory Keydir point lookup with average O(1) lookup cost.
• Logical-clock TTL records and lazy expiration.
• Atomic write batches with commit markers and restart recovery.
• Segment rolling and compaction with hint-file startup recovery.
• Ordered prefix/range scans with cursor pagination, page objects, bounded bulk deletion, storage statistics, verification, and deterministic keyspace fingerprints for ecosystem tools.
• A cloneable CLI for put, get, put-ttl, delete, delete-prefix, merge, scan, stats, verify, fingerprint, and benchmark flows.
• Native, JavaScript, and WebAssembly filesystem adapters.
• 24 first-party MoonBit source files, about 3,000 MoonBit source lines, and 63 first-party test declarations at the time of this acceptance review.

Scope and limitations

MoonKV is an embedded storage component, not a network server. The current scope is deliberately single-process and single-writer: applications should not open the same directory concurrently from multiple processes. Values are Bytes; the convenience CLI uses text values. TTL is a deterministic logical clock measured by database writes, not wall-clock time. The native and JavaScript adapters append directly; the WebAssembly adapter uses a portable read-merge-write fallback and is therefore not an append-throughput benchmark. The native adapter flushes each append, but this project does not claim filesystem fsync or power-loss durability.

User keys may be empty, but keys beginning with __tx__: or __tx_commit__: are reserved for the transaction journal and are rejected. max_file_size and TTL values must both be greater than zero.

Quick start

Clone and validate the repository:

git clone https://github.com/wallll-wal6/MoonKV.git
cd MoonKV
moon update
moon fmt --check
moon check --target all --deny-warn
moon build --target all
moon test --target all --deny-warn

Use MoonKV as a dependency in another MoonBit module:

moon add wallll-wal6/moonkv

The package metadata declares the public module name, repository, Apache-2.0 license, and the moonbitlang/x@0.4.46 dependency. The full package guide is in README.mbt.md.

Library API

import {
  "wallll-wal6/moonkv" @moonkv,
}

let db = @moonkv.DB::open("my_db", 10 * 1024 * 1024)
db.put("user_name", @moonkv.string_to_bytes("Alice"))
let value = db.get("user_name")
println(@moonkv.bytes_to_string(value))
db.close()

DB::open, put, get, delete, put_ttl, merge, and batch commit may raise MoonKVError. A caller should handle KeyNotFound, InvalidRecord, CorruptedDatabase, and IOError at its application boundary.

For cache indexes, queues, and local metadata services, the query API provides deterministic ordered reads without exposing internal transaction records:

let page = db.scan_prefix("user:", 100)
let next_page = db.scan_after("user:", page[page.length() - 1].0, 100)
let page_object = db.scan_page("user:", "", 100)
let stats = db.stats()
let fingerprint = db.fingerprint()

scan_range uses a half-open [start_key, end_key) interval. scan_page returns rows, has_more, and an optional continuation cursor. Scan limits must be positive; expired and deleted keys are omitted. delete_prefix uses a single atomic batch and is bounded by its limit. fingerprint is a stable FNV-1a smoke-check value, not a cryptographic integrity boundary.

CLI usage

The CLI uses a 10 MiB default segment size:

moon run cmd/main -- ./data_dir put mykey "Hello MoonBit"
moon run cmd/main -- ./data_dir get mykey
moon run cmd/main -- ./data_dir put-ttl token "temporary" 2
moon run cmd/main -- ./data_dir delete mykey
moon run cmd/main -- ./data_dir merge
moon run cmd/main -- ./data_dir scan user: 100
moon run cmd/main -- ./data_dir delete-prefix session: 100
moon run cmd/main -- ./data_dir stats
moon run cmd/main -- ./data_dir verify
moon run cmd/main -- ./data_dir fingerprint

The deterministic native benchmark writes and reads the requested number of records in one process. The scripts measure the complete command, including startup and filesystem overhead:

bash scripts/benchmark.sh 10000
# Windows PowerShell:
.\scripts\benchmark.ps1 -Records 10000

See benchmarks/README.md for methodology and the checked-in baseline format.

Storage and recovery model

Each record has a 28-byte header containing a checksum, logical timestamp, expiration timestamp, key size, and value size, followed by key and value bytes. Compaction rewrites live records to an immutable data segment and a matching hint file. It also creates the next empty active segment before returning, so records written by a later process cannot be appended to the hint-backed segment. The regression in scripts/cross-process-recovery.sh covers this restart sequence.

Verification and CI

The workflow in .github/workflows/test.yml runs on Linux, macOS, and Windows. It first attempts MoonBit 0.10.3. Because old 0.10.3 archives can be removed from the official distribution service, the workflow falls back to the current official toolchain only when that exact archive returns an error; the fallback is printed in the job log. It then updates dependencies, checks formatting, checks and builds every target, runs tests with warnings denied, runs the native cross-process recovery regression where the shell is available, and verifies that moon info leaves no generated-interface diff.

Local native validation additionally requires a C compiler. The full local equivalent is:

moon version --all
moon update
moon fmt --check
moon check --target all --deny-warn
moon build --target all
moon test --target all --deny-warn
bash scripts/cross-process-recovery.sh
moon info
git diff --exit-code

Repositories and project history

• GitHub: https://github.com/wallll-wal6/MoonKV
• GitLink mirror: https://gitlink.org.cn/wallll/moonkv
• OSC2026: https://moonbitlang.github.io/OSC2026/
• Development history: CHANGELOG.md
• Contribution process: CONTRIBUTING.md
• Dependency and attribution notes: NOTICE and docs/source-attribution.md

License

MoonKV is licensed under the Apache License 2.0. See LICENSE.