kscope

eBPF-based system anomaly observation and root-cause localization.

简体中文

Quick Start

./scripts/setup.sh                     # install deps, generate vmlinux.h
make release                           # optimized build
sudo ./target/release/kscope diagnose  # 60s idle observation

Demo Video

You can preview or download the demo video from Tencent weiyun.

Usage/Help

Use kscope -h or kscope <command> -h to see the full help page.

$ kscope -h
eBPF-based system anomaly observation and root-cause localization

Usage: kscope [OPTIONS] <COMMAND>

Commands:
  exectrace  Trace process exec() and print pid/ppid/comm
  diagnose   Observe the system for a fixed duration and produce a diagnostic report
  help       Print this message or the help of the given subcommand(s)

Options:
  -v, --verbose...  Log verbosity: -v = debug, -vv = trace. Overridden by RUST_LOG
  -h, --help        Print help
  -V, --version     Print version

Especifically, use kscope diagnose -h to see the detailed parameter description of the diagnostic subcommand:

$ kscope diagnose -h
Observe the system for a fixed duration and produce a diagnostic report

Usage: kscope diagnose [OPTIONS]

Options:
  -d, --duration <DURATION>        Observation duration in seconds [default: 60]
  -v, --verbose...                 Log verbosity: -v = debug, -vv = trace. Overridden by RUST_LOG
  -s, --scenario <SCENARIO>        Anomaly category to observe. "all" runs every available detector [default: all] [possible values: all, cpu, io, memory, lock, syscall]
      --sample-freq <SAMPLE_FREQ>  Per-CPU sampling frequency in Hz (CPU detector only) [default: 99]
  -o, --output-md <OUTPUT_MD>      Write Markdown report to this file (optional)
      --self-profile               Measure and report the tool's own CPU/memory overhead
      --llm                        Use LLM  to generate a natural-language summary. Requires `LLM_API_KEY` in the environment.  Falls back to the rule-based report if the key is missing or the API call fails
  -h, --help                       Print help

Examples

# Full observation (all 5 detectors)
sudo kscope diagnose -d 60 -s all -o report.md

# Single detector
sudo kscope diagnose -d 30 -s memory -o report.md

# With self-profiling overhead
sudo kscope diagnose -d 60 -s cpu --self-profile

# LLM HTML report (any OpenAI-compatible API)
env LLM_API_KEY=sk-xxx sudo -E kscope diagnose -d 60 -s all -o report.md --llm

# Process exec() tracer
sudo kscope exectrace

Scenarios: cpu, io, memory, lock, syscall, all

Reproduce anomalies for testing:

./scripts/repro_cpu.sh       # CPU stress
./scripts/repro_io.sh        # I/O stress
./scripts/repro_mem.sh       # memory thrashing
./scripts/repro_lock.sh      # lock contention
./scripts/repro_syscall.sh   # syscall latency

./scripts/run_all.sh         # idle observation
./scripts/run_all.sh stress  # all-in-one stress test
./scripts/run_all.sh stress llm 120  # stress + LLM report, 120s

LLM Report

When --llm is passed, kscope sends raw metrics to an LLM and generates an HTML report. Falls back to Markdown if the API key is missing.

# Via environment:
env LLM_API_KEY=sk-xxx sudo -E kscope diagnose -d 60 -s all -o report.md --llm

# Or create a .env file in the project root:
echo 'LLM_API_KEY=sk-xxx' > .env
sudo -E kscope diagnose -d 60 -s all --llm

Defaults to DeepSeek. Works with any OpenAI-compatible endpoint (LLM_BASE_URL, LLM_MODEL).

Requirements

• Linux kernel 6.6+ with BTF (CONFIG_DEBUG_INFO_BTF=y)
• root or CAP_BPF + CAP_PERFMON
• clang, llvm, bpftool, libelf-dev, zlib1g-dev, pkg-config, make
• Rust 1.75+

Tested on Debian 13 (x86_64, ARM64), openKylin 2.0 SP2 (x86_64).

License

GPL-2.0