PDServe:????SLO ??? PD ??????

PDServe ??? openEuler ? Prefill/Decode ??????????????Prefill?Decode????????????????,?? GPU ?????????????????????? Benchmark;?? GPU ?,?? vLLM/NIXL ??? kv_transfer_params ????? Decode ???

????:??????????Docker/Kubernetes ??????????????????????????????????? mock ??,??? GPU ??????????? docs/deployment.md ??? GPU ????

????

1P1D / XPYD ????:Prefill?Decode ????????????
SLO-aware ??:?? EWMA ??????????????????????????? SLO ???????
????:?? ID ???SQLite WAL ?????????????????????;Decode ?? KV ?????????????? Prefill ??????????
KV ?????:???????TTL ??????;?????????????????????
??????:????? PID ? pdserve- ??????????,?? dry-run,?????????
????:OpenAI ?????? TTFT?TPOT?E2E?????? SLO ??;?? Prometheus ?????
?????????:??????? mock ???? CI;vLLM/NIXL ????? Qwen3.6 Dense?Qwen3.6 MoE?DeepSeek-V4 ???? smoke profile?

?????

?? Python 3.11+,??????????? Python ??

python3 scripts/run_mock_cluster.py

????????????????:python3 scripts/run_mock_cluster.py --smoke?

????????:

curl http://127.0.0.1:8000/v1/completions \
  -H "Content-Type: application/json" \
  -d '{
    "request_id": "demo-001",
    "model": "mock-dense",
    "prompt": "Explain prefill/decode disaggregation",
    "max_tokens": 16,
    "slo": {"ttft_ms": 1200, "tpot_ms": 80, "deadline_ms": 15000}
  }'

????????????:

python3 -m unittest discover -s tests -v
python3 -m benchmark.selftest --requests 30 --concurrency 6

openEuler ????

?????? openeuler/openeuler:24.03-lts-sp4,?? root ????:

docker compose up --build -d
curl http://127.0.0.1:8000/health
docker compose down

2 ? NVIDIA GPU ? vLLM/NIXL ??:

MODEL=Qwen/Qwen3-0.6B \
docker compose -f docker-compose.yml -f docker-compose.vllm.yml up --build

???????UCX/NIXL ???????? ?????vLLM ??????,PD ???????????? TTFT ? token ???,??????????;?????????????????

4 ? GPU ?? 2-way TP ? Qwen3.6 MoE profile:

MODEL=Qwen/Qwen3.6-35B-A3B \
docker compose -f docker-compose.yml -f docker-compose.vllm.yml \
  -f docker-compose.vllm-4gpu.yml up --build

?? Dense/MoE ??????? configs/model-matrix.toml ???????????:

python3 -m benchmark.model_matrix --list
python3 -m benchmark.model_matrix --profile qwen36_moe --sample-gpu

?????

????? mock ???30 ????? 6?? 128 ?? token / 16 ?? token:

??	???	???	P99 E2E	P99 TTFT	??
1P1D ??	100%	5.11 req/s	1444.80 ms	191.02 ms	0
2P2D SLO ??	100%	10.00 req/s	790.45 ms	188.95 ms	0
2P2D + Prefill ??	100%	5.36 req/s	1156.92 ms	193.89 ms	3

??? mock ???,2P2D ?? 1P1D ?????? 95.8%,P99 E2E ??? 45.3%?????? 100% ?????,??? Prefill ??????????,????????? JSON ?? results/example-*.json,????? Benchmark ???

??? 5 ? Prefill+Decode ????????? 50/50 ??,?? 26 ???? 1 ??? KV ??????;? results/example-soak.json??????????????,??????? mock ???

????

pdserve/                 ????????????????
benchmark/               ????????????
configs/                 TOML ????
deploy/                  Kubernetes ????????
docs/                    ?????????API ?????
scripts/                 ?? 2P2D ???????
tests/                   ????? HTTP ?????
docker-compose.yml       openEuler CPU/mock ??
docker-compose.vllm.yml  vLLM/NIXL GPU ????
docker-compose.vllm-4gpu.yml  Dense/MoE 2-way TP ? GPU ????

???????

???	????
?????	2P2D ???OpenAI ?? API?KV ?????????????????????
????	??? loadgen???/?????TTFT/TPOT/P99/SLO????? JSON
?????	???????????????????????????????
????	??????API????Benchmark ???????

????????

????	??????????
????????	`Dockerfile` ?? openEuler 24.03 LTS SP4,? root ??
PD ?????	Router?Prefill?Decode?????????;Compose 2P2D,Kubernetes ? Deployment/HPA
Dense ? MoE ????	`configs/model-matrix.toml` ?????? ID;`benchmark/model_matrix.py` ?????????? SLO ??
??????	WAL?lease???????????????KV ??????;`tests/test_integration.py`
?????????	?? init??? IPC?NCCL watchdog?PID/??????? reaper???????
??? SLO ??	??/EWMA/??/SLO-aware ??;TTFT?TPOT?P99?goodput????? GPU ?????
KVCache ????	NIXL/UCX ??????????TTL/generation?????????????????
????????	`benchmark.selftest`?`benchmark.soak`?`benchmark.model_matrix` ??? JSON

??

???

Mulan PSL v2,? LICENSE?