MoonMDPKit

MoonMDPKit is a small, tabular Markov decision process toolkit written in MoonBit.

The project focuses on the classic dynamic-programming core of planning:

• finite states and finite actions
• sparse transition lists with explicit probabilities and rewards
• policy evaluation
• value iteration
• policy iteration
• a tiny CLI demo and two ready-to-run examples

This repo is intentionally scoped as a foundation rather than a full reinforcement-learning framework. That keeps the boundary clear and makes the code easier to audit, extend, and use as a reference implementation.

Competition links

• GitHub: https://github.com/zgfzgf899/MoonMDPKit
• GitLink: https://gitlink.org.cn/zgfzgf899/moonmdpkit

Authorship

The repository is maintained by a single contributor identity so the submission history stays clean and easy to review.

Why this shape

The competition rubric rewards work that is easy to reproduce and easy to grow. A tabular MDP library has a clean engineering boundary, well-known algorithms, and a natural path for future expansion into heuristics, stochastic shortest path problems, and richer environment adapters.

Build

moon test
moon run cmd/main

What is included

• MDP for finite tabular models
• Policy for deterministic action selection
• evaluate_policy for Bellman policy evaluation
• value_iteration for optimal value and policy search
• policy_iteration for alternating evaluation and improvement
• chain_example() and gridworld_example() as reference environments

Project layout

• moonmdpkit.mbt - top-level constants and project summary
• types.mbt - shared type aliases and result structs
• mdp.mbt - MDP model and transition storage
• policy.mbt - policy representation and helpers
• solvers.mbt - dynamic-programming algorithms
• examples.mbt - reusable example MDPs
• cmd/main/main.mbt - CLI demo
• moonmdpkit_wbtest.mbt - white-box tests

Usage sketch

let mdp = chain_example()
let result = value_iteration(mdp, DEFAULT_GAMMA, DEFAULT_TOLERANCE, 100)

println(result.iterations)
println(result.policy.action_at(0))
println(result.policy.action_at(1))

Verification

• moon test
• moon run cmd/main

Source notes

This repository was authored from scratch for the competition. The algorithms are standard tabular planning methods based on the classic definitions of MDPs, value iteration, and policy iteration. No third-party implementation was copied in.

License

Apache-2.0