FlagFFT is a JIT-compiled GPU FFT library. It generates backend-targeted GPU
kernels at runtime via Triton/TLE and
libtriton_jit, targeting
arbitrary-length transforms that vendor FFT libraries may not optimally
support. The current CMake build supports CUDA, MUSA, PPU, IX
(Iluvatar/Tianshu), MACA (MetaX), NPU (Ascend), and HCU (Hygon BW1000)
backends.
Build the library, install the Python codegen package, and run the full test
suite:
# 1. Clone
git clone https://github.com/flagos-ai/FlagFFT.git
cd FlagFFT
# 2. Initialize submodule
git submodule update --init --recursive
# 3. Build the library, CLI, and test binaries
cmake -B build -DCMAKE_BUILD_TYPE=Release \
-DFLAGFFT_BUILD_CLI=ON \
-DFLAGFFT_BUILD_TESTS=ON
cmake --build build -j$(nproc)
# 4. Install the Python codegen package (required for JIT kernel generation)
pip install .
# 5. Run the full accuracy + performance test suite
python tools/run_tests.py --combination full --gpus 0
The runner prints a live progress table and writes summary.json as a flat
array with one element per operator: accuracy (pass/fail), performance
(geometric mean speedup vs the selected backend’s reference FFT library) and
the path of the operator’s console log. The result directory holds nothing but
JSON and logs.
Docker
A CUDA development environment with the default dependencies is available:
docker build -t flagfft-dev -f docker/Dockerfile .
docker run --gpus all -v $(pwd):/workspace/FlagFFT -it flagfft-dev
# Inside the container, run steps 3-5 from above.
The Docker image and CI configuration use Python 3.12. MUSA, PPU, IX, MACA,
NPU, and HCU builds require their corresponding vendor SDK/runtime environment
and should be configured with -DBACKEND=MUSA, -DBACKEND=PPU,
-DBACKEND=IX, -DBACKEND=MACA, -DBACKEND=NPU, or -DBACKEND=HCU.
Dependencies
Required
Dependency
Minimum Version
Notes
CMake
3.18
Build system
C++ compiler
C++20 support
GCC 11+, Clang 14+
Python
3.10
JIT codegen + test runner; the provided CUDA Docker/CI environments use 3.12
flagtree
0.5.0
triton TLE support
SQLite3
—
Tuning database
Backend SDK
—
CUDA Toolkit, MUSA SDK, PPU SDK, CoreX CUDA-compatible SDK for IX, MACA SDK, CANN/Ascend runtime for NPU, or DTK 26.04/HIP + hipFFT for HCU
libtriton_jit
submodule
Triton JIT compiler (deps/libtriton_jit)
PyYAML
—
Test runner (pip install pyyaml)
Optional
Dependency
Purpose
Google Test
C++ unit tests (auto-fetched via FetchContent when FLAGFFT_BUILD_TESTS=ON)
Ninja
Faster build backend (cmake -G Ninja)
pytest
Python codegen tests
Submodule
Initialize the required submodule before building:
git submodule update --init --recursive
This pulls in deps/libtriton_jit, which provides the Triton JIT compiler and
nlohmann_json.
The default backend is CUDA. Select another supported backend at configure
time, for example -DBACKEND=MUSA, -DBACKEND=PPU, -DBACKEND=IX,
-DBACKEND=MACA, -DBACKEND=NPU, or -DBACKEND=HCU; the
corresponding SDK and runtime libraries must be installed.
The IX backend uses the CoreX CUDA-compatible driver/runtime and ixfft
reference library. It also requires an Iluvatar-enabled FlagTree/Triton
runtime (for example flagtree===0.5.1+iluvatar3.1).
MetaX C550 (MACA) Build
Use the tested MetaX container and the SDK’s cmake_maca / make_maca
wrappers. Set -DBACKEND=MACA -DMACA_PATH=/opt/maca; set all three device
filters to the physical card being tested. The card-4 build and validation
procedure is recorded with the MACA validation results in the workspace
results/ tree (20260917_165501_maca_hw_profile/REPORT.md).
Ascend (NPU) Build
Use the CANN 9 toolchain and the Ascend-enabled Triton/libtriton_jit checkout:
The FlagFFT Ascend 910B profile is FP32 C2C, R2C, and C2R for
contiguous 1D, 2D, and 3D plans. Native FP64 is unavailable on Ascend 910B:
Z2Z, Z2D, and D2Z plans return FLAGFFT_NOT_SUPPORTED; they are never
silently downcast or moved to a CPU fallback. The unified acceptance runner
omits the FP64 dtype cases on NPU before calculating totals and averages.
The native comparison and performance baseline uses the SiP FFT library from
the CANN NNAL installation (ASCEND_SIP_ROOT can override the default
/usr/local/Ascend/nnal/asdsip/latest). SiP accepts device tensors and covers
FP32 C2C/R2C/C2R in contiguous 1D/2D/3D out-of-place plans on 910B.
Each dimension must be at most 2^27 with no prime factor above 199, and the
documented input-element limit is 2^30. The runner still checks FlagFFT
against NumPy when SiP is unavailable for a case; only its platform and
performance rows are Skipped. SiP may modify the input of a power-of-two
horizontal 1D transform of length at least 32768, so the benchmark refreshes
its reference input before each execution.
When built with -DFLAGFFT_NPU_ENABLE_SIP_EXECUTION=ON (on by default for
BACKEND=NPU), FlagFFT also exposes FLAGFFT_NPU_EXECUTION_BACKEND=auto|sip|triton.
The default auto mode delegates eligible contiguous, out-of-place 1D FP32
plans to the installed SiP library and falls back to FlagFFT’s Triton executor
for unsupported shapes or in-place calls. sip requires SiP support, while
triton forces the independent FlagFFT kernels. Plan descriptions include the
selected execution backend. Since auto uses the same SiP library as the
benchmark reference, its speedup measures wrapper/delegation overhead rather
than the performance of FlagFFT’s own FFT kernels; use triton for native
kernel comparisons.
Hygon BW1000 (HCU) Build
Use the HCU 3.6 DTK 26.04 image from the
FlagTree HCU user manual.
The FlagFFT HCU adaptor uses HIP for device memory, streams, events, and
graphs, and hipFFT as the correctness/performance reference. BW1000 reports
gfx936 with 64-thread wavefronts, so the generated target is
hcu:gfx936:64. Set HIP_VISIBLE_DEVICES to the physical BW1000 IDs under
test; the acceptance runner sets it per worker when --gpus is used:
Path to the Python interpreter used by JIT codegen (default: python3 from PATH); keep its Python minor version aligned with the CMake build interpreter
FLAGFFT_TUNE_DB
Path to the SQLite tuning database (default: .flagfft/tuned_plans.sqlite beside the executable)
FLAGFFT_TUNE_DISABLE
Set to 1 to disable tuned plan lookup and always use auto-selected plans
On IX arch 71, contiguous FP32 1D single requests select scoped policies:
Length
Complex FFT policy
Real FFT policy
1024
[16,8,8], tensor exchange, two physical warps
Same leaf with real input/output
2048
Tensor exchange, two physical warps
R2C packs even/odd samples into a 1024-point complex leaf and reconstructs the half spectrum in one kernel; C2R keeps the real-boundary leaf
16384
Tensor exchange, inner pack 4, two physical warps
Same Four-Step with real input/output
328050, 340200
Interleaved shared exchange and twiddle recurrence
Half-length complex FFT plus pre/postprocess
663000
Unchanged
Half-length complex FFT plus pre/postprocess
1048576
Interleaved/swizzled exchange, inner pack 8, four physical warps
Half-length complex FFT plus pre/postprocess
For IX arch 71, contiguous FP32 1D batch-64 requests at length 1024, 2048,
or 16384 use tensor exchange and four physical warps. The 1024-point leaf uses
[8,8,4,4]; the 2048-point C2C/C2R leaf uses its measured factor order.
The 2048-point R2C path uses the fused 1024-point [16,8,8] leaf with two
physical warps. The same exchange policy covers the 8191-point batch-64
Bluestein plan, whose convolution uses a 16384-point Four-Step child.
For 23-point batch-64 R2C/C2R, a direct real DFT folds the two conversion
kernels into the transform; FLAGFFT_IX_REAL_DIRECT_DFT=0 restores the prior path.
At length 16384, batch-64 R2C/C2R use a packed 8192-point complex child;
FLAGFFT_PACKED_REAL=0 disables this real-transform path.
At lengths 185640, 340200, 524288, and 663000, batch-64 R2C/C2R also use
the packed half-length complex Four-Step child. These cases keep the full-length
path when FLAGFFT_IX_CT_BATCH=0 or FLAGFFT_PACKED_REAL=0.
Set FLAGFFT_IX_FUSED_R2C=0 before starting the process to
restore the prior full-length R2C leaf. Set FLAGFFT_IX_CT_BATCH=0 to
compare with the previous batch path.
Other lengths, batch sizes, ranks, architectures and FP64 retain their existing
paths. Set FLAGFFT_IX_CT_SINGLE=0before starting the process to compare
against the original single implementation. FLAGFFT_PACKED_REAL=0 can separately
disable the half-length real path. The policies select matching stage-twiddle
tables and have separate in-process and filesystem kernel cache entries.
Use tools/ix_ct_single_sweep.py --binary <build>/flagfft-cli --batches 1,64 --output-dir <workspace>/results/<timestamp>_ix_ct_single for serial, alternating
baseline/default measurements on physical GPU 2 (--gpu selects another card).
Accuracy validation remains
the responsibility of tools/run_tests.py; screening timings alone are not
acceptance results.
flagfft-cli device-info --json reports the current device’s driver-queried
warp size, thread-block limit and shared-memory limits. Code generation receives
these facts explicitly. balanced uses the device warp width for leaf launch
heuristics and bounds packing by queried shared-memory and live-value limits.
native uses the device warp width without the live-value packing guard.
Algorithmic
radices and transpose tile dimensions retain their existing meaning.
packed additionally targets one device warp when packing small leaf FFTs;
it is an experiment, not a claim of better performance for every shape.
Generated modules use profile-specific directories, and compiled plan text
records warp size, block threads, packing and the profile identifier.
Run isolated FP64 diagnostics and correctness-gated paired experiments in the
backend’s container, with PYTHONPATH pointing at this checkout’s python/:
Use a fresh timestamped output directory for each run. The paired experiment
uses warmup 5 / iterations 20, rotates policy order across repetitions, checks
FlagFFT against NumPy, and writes incremental CSV plus per-case logs. Times
cover the complete FFT execution, excluding plan creation/JIT and host copies.
For acceptance use tools/run_tests.py; the experiment is a representative
matrix, not a replacement for the complete 30-operator report.
FP64 diagnostics test native-SDK arithmetic (CoreX clang++, otherwise nvcc), Triton arithmetic and small
FlagFFT/platform FFTs in separate bounded processes. A failed compiler or
library probe does not prove missing hardware support; passing a small probe
does not certify every transform. Diagnostics do not enable IX FP64 acceptance.
Pass --capability-report <probe-dir>/capabilities.json to tools/run_tests.py
to attach the probe to the acceptance JSON. The runner checks the device identity
and launch limits before accepting this report; its recorded environment and
tested scope remain relevant when interpreting the results.
Install
cmake --install build --prefix /usr/local
Installs libflagfft.so, the public header (flagfft.h), and flagfft-cli (if
built), plus the private libflagfft_triton_jit.so runtime and the Python
helper scripts it needs, so an installed build does not depend on the source
checkout or on a separately installed libtriton_jit. The flagfft_codegen
Python package (pip install .), PyTorch, and the CUDA runtime remain external
prerequisites.
API
FlagFFT exposes a cuFFT-compatible C API in include/flagfft.h.
flagfftSetStream(plan, stream) // Attach a backend stream
flagfftDestroy(plan) // Free plan resources
flagfftGetPlanDescription(plan) // Human-readable plan summary
Data Types
FlagFFT Type
C Type
Description
flagfftComplex
float2
Single-precision complex
flagfftDoubleComplex
double2
Double-precision complex
flagfftReal
float
Single-precision real
flagfftDoubleReal
double
Double-precision real
Transform Types
Type Constant
Transform
FLAGFFT_C2C
Complex → Complex
FLAGFFT_Z2Z
Double Complex → Double Complex
FLAGFFT_R2C
Real → Complex
FLAGFFT_D2Z
Double Real → Double Complex
FLAGFFT_C2R
Complex → Real
FLAGFFT_Z2D
Double Complex → Double Real
flagfftPlan3d supports contiguous row-major rank-3 plans for all six
transform types. The corresponding contiguous rank-3 forms are also available
through flagfftPlanMany; arbitrary custom strides and layouts beyond the
supported forms remain unsupported.
✅ FlagFFT C2C, R2C, C2R for contiguous 1D/2D/3D; FP64 Z2Z/Z2D/D2Z return FLAGFFT_NOT_SUPPORTED; SiP is the FP32 platform reference
Batched transforms
✅
In-place and out-of-place
✅
Backend adaptors
✅ CUDA, MUSA, PPU, IX, NPU (selected at build time)
Backend stream attachment
✅
For the CUDA backend, 2^20 rank-1 transforms on sm_80 select a 1024 x 1024
Four-Step decomposition. The TLE kernels apply contiguous, asynchronously
loaded twiddles in the row pass, pack two adjacent row FFTs and four adjacent
column FFTs per block for single precision, and XOR-swizzle shared-memory
indices to reduce bank conflicts.
Planned / Not Yet Supported
Feature
Status
Rank-2 more exec algos
RTRT only now
CLI Tool
flagfft-cli is a native benchmark and verification tool. Build it with
-DFLAGFFT_BUILD_CLI=ON.
tune builds the rank-1 decomposition candidates for one length, screens
--max-candidates of them, re-times the best --finalists, validates each
candidate’s output against the reference, and persists one validated winner
per request (device architecture, length, batch bucket, dtype and direction)
into the SQLite tuning database. --api accepts c2c or z2z today, and
--no-save runs the same search without writing to the database.
Option
Default
Description
--shape
1048576
Single 1D FFT length
--batch
1
Batch size
--api
c2c
Complex precision: c2c or z2z
--max-candidates
5
Candidate plans to screen
--finalists
2
Candidates that get the long benchmark
--screen-warmup / --screen-iters
10 / 50
Screening timings
--final-warmup / --final-iters
50 / 1000
Finalist timings
--db
.flagfft/tuned_plans.sqlite beside the executable
Tuning database
--no-save
—
Do not persist trials or the winning plan
--json
—
Output results as JSON
Exit Codes
Code
Meaning
0
Passed
1
Failed / invalid arguments
2
Runtime error
77
Skipped / unsupported
Testing
FlagFFT has three layers of testing: a unified Python test runner, C++ unit
tests (Google Test), and Python codegen tests (pytest).
Unified Test Runner
tools/run_tests.py is the single entry point for the 30 acceptance operators.
Correctness compares FlagFFT and, when available, the platform FFT library
independently against a float64/complex128 NumPy reference. An operator passes
correctness only when all selected FlagFFT-vs-NumPy cases pass; platform-library
failures are reported under platform_accuracy and do not fail FlagFFT
correctness. Performance continues to use flagfft-cli bench.
Install test dependencies with pip install -e '.[test]' and build with
-DFLAGFFT_BUILD_TESTS=ON -DFLAGFFT_BUILD_CLI=ON. The test build includes
build/ctest/numpy_fft_capture; no separate validation build is required.
conf/operators.yaml defines the following 30 logical operators. Each operator
expands to torch.float32 and torch.float64; the runner maps those dtype
values to C2C/R2C/C2R or Z2Z/D2Z/Z2D internally.
Group
Operator ID example
1D CT single
1d_ct_single_c2c
1D CT batch
1d_ct_batch_c2c
1D four-step single
1d_fourstep_single_c2c
1D four-step batch
1d_fourstep_batch_c2c
1D prime single
1d_prime_single_c2c
1D prime batch
1d_prime_batch_c2c
2D single
2d_c2c
3D single
3d_c2c
2D batch
2d_batch_c2c
3D batch
3d_batch_c2c
batch: single/batch in an operator is a category, not a numeric batch
count. conf/test_matrix.yaml sets 1D batch to 64, 2D/3D batch to 4, and keeps
single-transform batch at 1. All cases run out-of-place, including 2D/3D batch
cases through contiguous PlanMany layouts without custom embeds or strides.
The current full matrix expands to 400 cases before backend dtype omissions.
The four-step group also checks that the captured runtime plan contains a
FourStep node.
The current IX acceptance policy disables FP64. The runner omits
torch.float64 cases from the manifest’s expanded case list on IX and does
not count them as skipped or completed work; every logical operator still runs
for FP32.
On NPU, Ascend 910B does not support FP64, so the runner omits those cases
before launch and aggregation. The FP32 cases run FlagFFT against NumPy. SiP
is used for its supported out-of-place 1D/2D/3D C2C/R2C/C2R shapes. Cases
outside SiP’s size or prime-factor limits retain the FlagFFT NumPy result while
their platform and performance rows are marked Skipped. NPU performance
reports SiP reference timing and speedup wherever that reference exists.
Complex APIs test both directions; real-to-complex APIs test forward and
complex-to-real APIs test inverse. Real-inverse inputs have valid
multidimensional Hermitian half spectra. NumPy inverse results are multiplied
by the transform size to match the unnormalized device APIs. Both comparisons
use the existing size/precision-aware worst-batch rel_l2 and rel_linf
limits and reject nonfinite values.
Usage
Flag
Default
Description
--ops
All 30
Comma-separated acceptance operator IDs
--op-list-file
—
One operator ID per line; # starts a comment
--start
—
Start at this operator in configuration order
--combination
full
Group filter: the six groups above; comma-separated, or full/all alone
--gpus
0
Comma-separated device IDs or all
--build-dir
build beside the runner
CMake build directory
--capture-bin
<build-dir>/ctest/numpy_fft_capture
Optional native capture override
--output-dir
Workspace results/<timestamp>_acceptance30
Fresh result directory
--incremental-csv
<output-dir>/incremental.csv
One flushed row per completed case/phase
--accuracy-only / --performance-only
Both phases
Mutually exclusive phase filters
--scales
Matrix scales, or [1.0] if omitted
Comma-separated positive input amplitudes, or all for 2^-20,1,2^20
--shapes
All configured sizes
Exact shapes, e.g. 256,64x64
--max-cases
—
Select the first N cases for a partial run
--timeout
200
Independent timeout for each FlagFFT/platform/benchmark process
--warmup / --iters
10 / 100
Benchmark warmup and measurement iterations
--dry-run
—
Print selected cases without execution or result files
--color
auto
auto/always/never
The old 18 API/rank IDs are replaced by the acceptance IDs above. The old
1d_bs_single/batch group filters are accepted as aliases for
1d_prime_single/batch; 2D is now one group containing both CT and Prime
sizes. Stages are no longer configured or filtered.
# Full acceptance suite; results are outside the repository/worktree.
python tools/run_tests.py --gpus 0
# Inspect the entire 30-operator execution matrix without using a GPU.
python tools/run_tests.py --dry-run
# NumPy correctness for selected operators.
python tools/run_tests.py --accuracy-only \
--ops 1d_ct_single_c2c,1d_prime_batch_c2r
# Prime batch group.
python tools/run_tests.py --combination 1d_prime_batch --gpus 0
# All three correctness scales. Benchmark runs once per shape/batch/direction.
python tools/run_tests.py --scales all --ops 1d_ct_single_c2c
Output
All outputs use format version 3. summary.json is a flat array with one
element per operator, matching the shape the acceptance platform parses.
Filtered runs contain the selected operators, and the manifest records the
exact partial selection.
manifest.json: selected operators, complete expected case lists, parameter matrix and runtime environment.
summary.json: one element per operator, in acceptance order, carrying accuracy, platform_accuracy, performance, perf_log_path and the run metadata.
incremental.csv: case/phase, operator ID, shape, numeric batch, direction, scale, both correctness statuses and errors, policy skip reason, limits, timings and actual plan. CSV quoting preserves multiline plan text.
{op_id}/{accuracy,platform_accuracy,performance}_result.json: aggregate per-operator results, including the per-case metrics, seeds, hashes and plans. The accuracy.details and performance.data.default report fields are retained here.
{op_id}/accuracy.log: the FlagFFT and platform capture console output for every accuracy case of that operator, with each plan between FLAGFFT PLAN BEGIN/END delimiters and each case introduced by a ----- <case_id> <impl> ----- header. The file closes with a pytest-shaped verdict line, so a platform log parser derives the same counts and status as summary.json.
{op_id}/perf.log: the benchmark console output for every performance case.
Neither the generated input nor the captured output ever touches the disk. The
input is regenerated from its seed one batch group at a time and pushed into the
capture’s stdin as the capture consumes it, and the output is streamed back
through a pipe and folded into the NumPy comparison batch by batch. The scratch
directory holds nothing but the capture’s own console logs. No .bin, .npy,
case.json, flagfft_plan.txt or per-case directory is produced, so a full run
writes exactly five files per operator.
In summary.json, accuracy and platform_accuracy carry the counts, the
PASS/FAIL status and the log path the platform reads; performance is one
row per case with the measured speedup in the complex column and the operator’s
geometric mean in avg_speedup.
Correctness and performance have separate case IDs; scale is omitted from
performance IDs. A missing result cannot make an operator pass. Numeric
failures serialize nonfinite metrics as JSON null while retaining failure
status. A plan is saved as soon as its creation succeeds and refreshed after
successful execution; creation failures have plan: null.
Performance rows retain the raw speedup and record baseline_valid. NPU rows
for SiP-supported cases include reference timing and speedup; policy-skipped
reference cases serialize those fields as null.
An incorrect platform baseline is excluded from the aggregate speedup
statistics; operators failing FlagFFT correctness are excluded as well.
Performance-only runs have an unknown baseline validity.
Exit code is 0 when all requested FlagFFT correctness and/or performance phases
pass; unsupported fp64 dtypes are omitted from the totals. Exit code 1
indicates a failure, incomplete phase, or unexpected skip; 2 is a configuration
error and 130 is interruption. Platform correctness is reported independently.
C++ Tests (ctest/)
Built with -DFLAGFFT_BUILD_TESTS=ON. Each test binary compares FlagFFT
output against the selected backend’s reference FFT library using normwise
relative error metrics (rel_l2, rel_linf). On NPU, the reference-based
GTest binaries are skipped; ctest/numpy_fft_capture is used by the unified
runner. It runs FlagFFT for every selected FP32 case, compares against NumPy,
and invokes SiP for cases inside its support matrix.
Structure
Test Pattern
Coverage
test_plan
Plan lifecycle, error codes, unsupported API contracts
test_2d_correctness
Rank-2 all-API correctness and complex/real roundtrips
test_3d_correctness
Rank-3 all-API correctness and complex/real roundtrips
Suffix key: s = single-batch, b = multi-batch; ct = Cooley-Tukey, bs
= Bluestein/Rader.
The unified acceptance runner is the supported entry point for IX and NPU. It
keeps the three FP64 APIs in the report but filters them before native
execution. On NPU it also applies the SiP 1D/2D/3D support matrix per case:
FlagFFT accuracy continues to run, while an unavailable platform comparison or
benchmark is recorded as Skipped. Direct invocation of bundled FP64 C++ test
binaries is not an IX or NPU acceptance test and may call unsupported vendor
functionality. NPU FLAGFFT_BUILD_TESTS builds the capture executable and
links the installed SiP FFT library.
Running Individual Tests
# Run a specific test
./build/ctest/test_exec_c2c_fwd_ct_s
# With custom parameters
./build/ctest/test_exec_c2c_fwd_ct_s --nx 4096 --batch 64 --direction forward
# Run all ctest tests
cd build && ctest --output-on-failure
Test binaries accept the applicable subset of: --nx, --ny, --nz,
--batch, --direction, --api, --scale, --json-file.
Python Tests
Tests for the flagfft_codegen Python package. Requires the package installed
(pip install .).
# Run all Python tests
pytest tests/python/ -v
# Run only codegen-marked tests
pytest tests/python/ -v -m codegen
Tests cover codelet structure, kernel source generation, JIT CSV parsing, and
Bluestein/reshape/R2C metadata. Tests that require Triton/TLE are
automatically skipped when dependencies are unavailable.
Test Configuration
The test parameter space is defined in conf/:
conf/operators.yaml — the fixed 30-operator acceptance list and dtype pairs
conf/test_matrix.yaml — CT/four-step/prime and 2D/3D sizes, batch counts, and scales
License
Apache License, Version 2.0. See LICENSE for the full text.
FlagFFT
FlagFFT is a JIT-compiled GPU FFT library. It generates backend-targeted GPU kernels at runtime via Triton/TLE and libtriton_jit, targeting arbitrary-length transforms that vendor FFT libraries may not optimally support. The current CMake build supports CUDA, MUSA, PPU, IX (Iluvatar/Tianshu), MACA (MetaX), NPU (Ascend), and HCU (Hygon BW1000) backends.
Table of Contents
Quick Start
Build the library, install the Python codegen package, and run the full test suite:
The runner prints a live progress table and writes
summary.jsonas a flat array with one element per operator: accuracy (pass/fail), performance (geometric mean speedup vs the selected backend’s reference FFT library) and the path of the operator’s console log. The result directory holds nothing but JSON and logs.Docker
A CUDA development environment with the default dependencies is available:
The Docker image and CI configuration use Python 3.12. MUSA, PPU, IX, MACA, NPU, and HCU builds require their corresponding vendor SDK/runtime environment and should be configured with
-DBACKEND=MUSA,-DBACKEND=PPU,-DBACKEND=IX,-DBACKEND=MACA,-DBACKEND=NPU, or-DBACKEND=HCU.Dependencies
Required
deps/libtriton_jit)pip install pyyaml)Optional
FLAGFFT_BUILD_TESTS=ON)cmake -G Ninja)Submodule
Initialize the required submodule before building:
This pulls in
deps/libtriton_jit, which provides the Triton JIT compiler andnlohmann_json.Building
Basic Build (library only)
This produces
build/libflagfft.so.Build Options
FLAGFFT_BUILD_CLIOFFflagfft-clibenchmark/verification toolFLAGFFT_BUILD_TESTSOFFBACKENDCUDACUDA,MUSA,PPU,IX,MACA,NPU, orHCUCMAKE_BUILD_TYPERelease,Debug,RelWithDebInfoFull Build (library + CLI + tests)
The default backend is CUDA. Select another supported backend at configure time, for example
-DBACKEND=MUSA,-DBACKEND=PPU,-DBACKEND=IX,-DBACKEND=MACA,-DBACKEND=NPU, or-DBACKEND=HCU; the corresponding SDK and runtime libraries must be installed.Iluvatar/Tianshu (IX) Build
The IX backend uses the CoreX CUDA-compatible driver/runtime and ixfft reference library. It also requires an Iluvatar-enabled FlagTree/Triton runtime (for example
flagtree===0.5.1+iluvatar3.1).MetaX C550 (MACA) Build
Use the tested MetaX container and the SDK’s
cmake_maca/make_macawrappers. Set-DBACKEND=MACA -DMACA_PATH=/opt/maca; set all three device filters to the physical card being tested. The card-4 build and validation procedure is recorded with the MACA validation results in the workspaceresults/tree (20260917_165501_maca_hw_profile/REPORT.md).Ascend (NPU) Build
Use the CANN 9 toolchain and the Ascend-enabled Triton/libtriton_jit checkout:
The FlagFFT Ascend 910B profile is FP32
C2C,R2C, andC2Rfor contiguous 1D, 2D, and 3D plans. Native FP64 is unavailable on Ascend 910B:Z2Z,Z2D, andD2Zplans returnFLAGFFT_NOT_SUPPORTED; they are never silently downcast or moved to a CPU fallback. The unified acceptance runner omits the FP64 dtype cases on NPU before calculating totals and averages.The native comparison and performance baseline uses the SiP FFT library from the CANN NNAL installation (
ASCEND_SIP_ROOTcan override the default/usr/local/Ascend/nnal/asdsip/latest). SiP accepts device tensors and covers FP32C2C/R2C/C2Rin contiguous 1D/2D/3D out-of-place plans on 910B. Each dimension must be at most2^27with no prime factor above 199, and the documented input-element limit is2^30. The runner still checks FlagFFT against NumPy when SiP is unavailable for a case; only its platform and performance rows areSkipped. SiP may modify the input of a power-of-two horizontal 1D transform of length at least 32768, so the benchmark refreshes its reference input before each execution.When built with
-DFLAGFFT_NPU_ENABLE_SIP_EXECUTION=ON(on by default forBACKEND=NPU), FlagFFT also exposesFLAGFFT_NPU_EXECUTION_BACKEND=auto|sip|triton. The defaultautomode delegates eligible contiguous, out-of-place 1D FP32 plans to the installed SiP library and falls back to FlagFFT’s Triton executor for unsupported shapes or in-place calls.siprequires SiP support, whiletritonforces the independent FlagFFT kernels. Plan descriptions include the selected execution backend. Sinceautouses the same SiP library as the benchmark reference, its speedup measures wrapper/delegation overhead rather than the performance of FlagFFT’s own FFT kernels; usetritonfor native kernel comparisons.Hygon BW1000 (HCU) Build
Use the HCU 3.6 DTK 26.04 image from the FlagTree HCU user manual. The FlagFFT HCU adaptor uses HIP for device memory, streams, events, and graphs, and hipFFT as the correctness/performance reference. BW1000 reports
gfx936with 64-thread wavefronts, so the generated target ishcu:gfx936:64. SetHIP_VISIBLE_DEVICESto the physical BW1000 IDs under test; the acceptance runner sets it per worker when--gpusis used:Environment Variables
FLAGFFT_PYTHONpython3from PATH); keep its Python minor version aligned with the CMake build interpreterFLAGFFT_TUNE_DB.flagfft/tuned_plans.sqlitebeside the executable)FLAGFFT_TUNE_DISABLE1to disable tuned plan lookup and always use auto-selected plansFLAGFFT_EXECUTION_POLICYbalanced(IX default),native(HCU default; device warp with queried resource caps),legacy(other backends’ default/comparison), orpacked(experimental wider leaf packing)Hardware profiles and IX experiments
On IX arch
71, contiguous FP32 1D single requests select scoped policies:[16,8,8], tensor exchange, two physical warpsFor IX arch
71, contiguous FP32 1D batch-64 requests at length 1024, 2048, or 16384 use tensor exchange and four physical warps. The 1024-point leaf uses[8,8,4,4]; the 2048-point C2C/C2R leaf uses its measured factor order. The 2048-point R2C path uses the fused 1024-point[16,8,8]leaf with two physical warps. The same exchange policy covers the 8191-point batch-64 Bluestein plan, whose convolution uses a 16384-point Four-Step child. For 23-point batch-64 R2C/C2R, a direct real DFT folds the two conversion kernels into the transform;FLAGFFT_IX_REAL_DIRECT_DFT=0restores the prior path. At length 16384, batch-64 R2C/C2R use a packed 8192-point complex child;FLAGFFT_PACKED_REAL=0disables this real-transform path. At lengths 185640, 340200, 524288, and 663000, batch-64 R2C/C2R also use the packed half-length complex Four-Step child. These cases keep the full-length path whenFLAGFFT_IX_CT_BATCH=0orFLAGFFT_PACKED_REAL=0. SetFLAGFFT_IX_FUSED_R2C=0before starting the process to restore the prior full-length R2C leaf. SetFLAGFFT_IX_CT_BATCH=0to compare with the previous batch path.Other lengths, batch sizes, ranks, architectures and FP64 retain their existing paths. Set
FLAGFFT_IX_CT_SINGLE=0before starting the process to compare against the original single implementation.FLAGFFT_PACKED_REAL=0can separately disable the half-length real path. The policies select matching stage-twiddle tables and have separate in-process and filesystem kernel cache entries.Use
tools/ix_ct_single_sweep.py --binary <build>/flagfft-cli --batches 1,64 --output-dir <workspace>/results/<timestamp>_ix_ct_singlefor serial, alternating baseline/default measurements on physical GPU 2 (--gpuselects another card). Accuracy validation remains the responsibility oftools/run_tests.py; screening timings alone are not acceptance results.flagfft-cli device-info --jsonreports the current device’s driver-queried warp size, thread-block limit and shared-memory limits. Code generation receives these facts explicitly.balanceduses the device warp width for leaf launch heuristics and bounds packing by queried shared-memory and live-value limits.nativeuses the device warp width without the live-value packing guard. Algorithmic radices and transpose tile dimensions retain their existing meaning.packedadditionally targets one device warp when packing small leaf FFTs; it is an experiment, not a claim of better performance for every shape. Generated modules use profile-specific directories, and compiled plan text records warp size, block threads, packing and the profile identifier.Run isolated FP64 diagnostics and correctness-gated paired experiments in the backend’s container, with
PYTHONPATHpointing at this checkout’spython/:Use a fresh timestamped output directory for each run. The paired experiment uses warmup 5 / iterations 20, rotates policy order across repetitions, checks FlagFFT against NumPy, and writes incremental CSV plus per-case logs. Times cover the complete FFT execution, excluding plan creation/JIT and host copies. For acceptance use
tools/run_tests.py; the experiment is a representative matrix, not a replacement for the complete 30-operator report.FP64 diagnostics test native-SDK arithmetic (CoreX
clang++, otherwisenvcc), Triton arithmetic and small FlagFFT/platform FFTs in separate bounded processes. A failed compiler or library probe does not prove missing hardware support; passing a small probe does not certify every transform. Diagnostics do not enable IX FP64 acceptance. Pass--capability-report <probe-dir>/capabilities.jsontotools/run_tests.pyto attach the probe to the acceptance JSON. The runner checks the device identity and launch limits before accepting this report; its recorded environment and tested scope remain relevant when interpreting the results.Install
Installs
libflagfft.so, the public header (flagfft.h), andflagfft-cli(if built), plus the privatelibflagfft_triton_jit.soruntime and the Python helper scripts it needs, so an installed build does not depend on the source checkout or on a separately installedlibtriton_jit. Theflagfft_codegenPython package (pip install .), PyTorch, and the CUDA runtime remain external prerequisites.API
FlagFFT exposes a cuFFT-compatible C API in
include/flagfft.h.Plan Creation
Execution
Management
Data Types
flagfftComplexfloat2flagfftDoubleComplexdouble2flagfftRealfloatflagfftDoubleRealdoubleTransform Types
FLAGFFT_C2CFLAGFFT_Z2ZFLAGFFT_R2CFLAGFFT_D2ZFLAGFFT_C2RFLAGFFT_Z2DflagfftPlan3dsupports contiguous row-major rank-3 plans for all six transform types. The corresponding contiguous rank-3 forms are also available throughflagfftPlanMany; arbitrary custom strides and layouts beyond the supported forms remain unsupported.Currently Supported
FLAGFFT_NOT_SUPPORTED; SiP is the FP32 platform referenceFor the CUDA backend,
2^20rank-1 transforms onsm_80select a1024 x 1024Four-Step decomposition. The TLE kernels apply contiguous, asynchronously loaded twiddles in the row pass, pack two adjacent row FFTs and four adjacent column FFTs per block for single precision, and XOR-swizzle shared-memory indices to reduce bank conflicts.Planned / Not Yet Supported
CLI Tool
flagfft-cliis a native benchmark and verification tool. Build it with-DFLAGFFT_BUILD_CLI=ON.Subcommands
bench— Benchmark FFT performance--rank11,2, or3--apic2cc2c,z2z,r2c,d2z,c2r,z2d--shape1024,256x256,16x16x16,1024,2048,4096--batch1--directionforwardforwardorinverse--placementout-of-placeout-of-placeorin-place--warmup10--iters100--json--print-path--json)Examples:
tune— Decomposition auto-tuningtunebuilds the rank-1 decomposition candidates for one length, screens--max-candidatesof them, re-times the best--finalists, validates each candidate’s output against the reference, and persists one validated winner per request (device architecture, length, batch bucket, dtype and direction) into the SQLite tuning database.--apiacceptsc2corz2ztoday, and--no-saveruns the same search without writing to the database.--shape1048576--batch1--apic2cc2corz2z--max-candidates5--finalists2--screen-warmup/--screen-iters10/50--final-warmup/--final-iters50/1000--db.flagfft/tuned_plans.sqlitebeside the executable--no-save--jsonExit Codes
01277Testing
FlagFFT has three layers of testing: a unified Python test runner, C++ unit tests (Google Test), and Python codegen tests (pytest).
Unified Test Runner
tools/run_tests.pyis the single entry point for the 30 acceptance operators. Correctness compares FlagFFT and, when available, the platform FFT library independently against a float64/complex128 NumPy reference. An operator passes correctness only when all selected FlagFFT-vs-NumPy cases pass; platform-library failures are reported underplatform_accuracyand do not fail FlagFFT correctness. Performance continues to useflagfft-cli bench.Install test dependencies with
pip install -e '.[test]'and build with-DFLAGFFT_BUILD_TESTS=ON -DFLAGFFT_BUILD_CLI=ON. The test build includesbuild/ctest/numpy_fft_capture; no separate validation build is required.conf/operators.yamldefines the following 30 logical operators. Each operator expands totorch.float32andtorch.float64; the runner maps those dtype values to C2C/R2C/C2R or Z2Z/D2Z/Z2D internally.1d_ct_single_c2c1d_ct_batch_c2c1d_fourstep_single_c2c1d_fourstep_batch_c2c1d_prime_single_c2c1d_prime_batch_c2c2d_c2c3d_c2c2d_batch_c2c3d_batch_c2cbatch: single/batchin an operator is a category, not a numeric batch count.conf/test_matrix.yamlsets 1D batch to 64, 2D/3D batch to 4, and keeps single-transform batch at 1. All cases run out-of-place, including 2D/3D batch cases through contiguous PlanMany layouts without custom embeds or strides. The current full matrix expands to 400 cases before backend dtype omissions. The four-step group also checks that the captured runtime plan contains a FourStep node.The current IX acceptance policy disables FP64. The runner omits
torch.float64cases from the manifest’s expanded case list on IX and does not count them as skipped or completed work; every logical operator still runs for FP32.On NPU, Ascend 910B does not support FP64, so the runner omits those cases before launch and aggregation. The FP32 cases run FlagFFT against NumPy. SiP is used for its supported out-of-place 1D/2D/3D C2C/R2C/C2R shapes. Cases outside SiP’s size or prime-factor limits retain the FlagFFT NumPy result while their platform and performance rows are marked
Skipped. NPU performance reports SiP reference timing and speedup wherever that reference exists.Complex APIs test both directions; real-to-complex APIs test forward and complex-to-real APIs test inverse. Real-inverse inputs have valid multidimensional Hermitian half spectra. NumPy inverse results are multiplied by the transform size to match the unnormalized device APIs. Both comparisons use the existing size/precision-aware worst-batch
rel_l2andrel_linflimits and reject nonfinite values.Usage
--ops--op-list-file#starts a comment--start--combinationfullfull/allalone--gpus0all--build-dirbuildbeside the runner--capture-bin<build-dir>/ctest/numpy_fft_capture--output-dirresults/<timestamp>_acceptance30--incremental-csv<output-dir>/incremental.csv--accuracy-only/--performance-only--scales[1.0]if omittedallfor2^-20,1,2^20--shapes256,64x64--max-cases--timeout200--warmup/--iters10 / 100--dry-run--colorautoauto/always/neverThe old 18 API/rank IDs are replaced by the acceptance IDs above. The old
1d_bs_single/batchgroup filters are accepted as aliases for1d_prime_single/batch; 2D is now one group containing both CT and Prime sizes. Stages are no longer configured or filtered.Output
All outputs use format version 3.
summary.jsonis a flat array with one element per operator, matching the shape the acceptance platform parses. Filtered runs contain the selected operators, and the manifest records the exact partial selection.manifest.json: selected operators, complete expected case lists, parameter matrix and runtime environment.summary.json: one element per operator, in acceptance order, carryingaccuracy,platform_accuracy,performance,perf_log_pathand the run metadata.incremental.csv: case/phase, operator ID, shape, numeric batch, direction, scale, both correctness statuses and errors, policy skip reason, limits, timings and actual plan. CSV quoting preserves multiline plan text.{op_id}/{accuracy,platform_accuracy,performance}_result.json: aggregate per-operator results, including the per-case metrics, seeds, hashes and plans. Theaccuracy.detailsandperformance.data.defaultreport fields are retained here.{op_id}/accuracy.log: the FlagFFT and platform capture console output for every accuracy case of that operator, with each plan betweenFLAGFFT PLAN BEGIN/ENDdelimiters and each case introduced by a----- <case_id> <impl> -----header. The file closes with a pytest-shaped verdict line, so a platform log parser derives the same counts and status assummary.json.{op_id}/perf.log: the benchmark console output for every performance case.Neither the generated input nor the captured output ever touches the disk. The input is regenerated from its seed one batch group at a time and pushed into the capture’s stdin as the capture consumes it, and the output is streamed back through a pipe and folded into the NumPy comparison batch by batch. The scratch directory holds nothing but the capture’s own console logs. No
.bin,.npy,case.json,flagfft_plan.txtor per-case directory is produced, so a full run writes exactly five files per operator.In
summary.json,accuracyandplatform_accuracycarry the counts, thePASS/FAILstatus and the log path the platform reads;performanceis one row per case with the measured speedup in the complex column and the operator’s geometric mean inavg_speedup.Correctness and performance have separate case IDs; scale is omitted from performance IDs. A missing result cannot make an operator pass. Numeric failures serialize nonfinite metrics as JSON null while retaining failure status. A plan is saved as soon as its creation succeeds and refreshed after successful execution; creation failures have
plan: null.Performance rows retain the raw speedup and record
baseline_valid. NPU rows for SiP-supported cases include reference timing and speedup; policy-skipped reference cases serialize those fields asnull. An incorrect platform baseline is excluded from the aggregate speedup statistics; operators failing FlagFFT correctness are excluded as well. Performance-only runs have an unknown baseline validity.Exit code is 0 when all requested FlagFFT correctness and/or performance phases pass; unsupported fp64 dtypes are omitted from the totals. Exit code 1 indicates a failure, incomplete phase, or unexpected skip; 2 is a configuration error and 130 is interruption. Platform correctness is reported independently.
C++ Tests (ctest/)
Built with
-DFLAGFFT_BUILD_TESTS=ON. Each test binary compares FlagFFT output against the selected backend’s reference FFT library using normwise relative error metrics (rel_l2,rel_linf). On NPU, the reference-based GTest binaries are skipped;ctest/numpy_fft_captureis used by the unified runner. It runs FlagFFT for every selected FP32 case, compares against NumPy, and invokes SiP for cases inside its support matrix.Structure
test_plantest_2d_correctnesstest_3d_correctnesstest_exec_c2c_{fwd,inv}_{ct,bs}_{s,b}test_exec_z2z_{fwd,inv}_{ct,bs}_{s,b}test_exec_r2c_{ct,bs}_{s,b}test_exec_d2z_{ct,bs}_{s,b}test_exec_c2r_{ct,bs}_{s,b}test_exec_z2d_{ct,bs}_{s,b}test_exec_r2c_c2r_{ct,bs}_{s,b}test_exec_d2z_z2d_{ct,bs}_{s,b}Suffix key:
s= single-batch,b= multi-batch;ct= Cooley-Tukey,bs= Bluestein/Rader.The unified acceptance runner is the supported entry point for IX and NPU. It keeps the three FP64 APIs in the report but filters them before native execution. On NPU it also applies the SiP 1D/2D/3D support matrix per case: FlagFFT accuracy continues to run, while an unavailable platform comparison or benchmark is recorded as
Skipped. Direct invocation of bundled FP64 C++ test binaries is not an IX or NPU acceptance test and may call unsupported vendor functionality. NPUFLAGFFT_BUILD_TESTSbuilds the capture executable and links the installed SiP FFT library.Running Individual Tests
Test binaries accept the applicable subset of:
--nx,--ny,--nz,--batch,--direction,--api,--scale,--json-file.Python Tests
Tests for the
flagfft_codegenPython package. Requires the package installed (pip install .).Tests cover codelet structure, kernel source generation, JIT CSV parsing, and Bluestein/reshape/R2C metadata. Tests that require Triton/TLE are automatically skipped when dependencies are unavailable.
Test Configuration
The test parameter space is defined in
conf/:conf/operators.yaml— the fixed 30-operator acceptance list and dtype pairsconf/test_matrix.yaml— CT/four-step/prime and 2D/3D sizes, batch counts, and scalesLicense
Apache License, Version 2.0. See LICENSE for the full text.