[Triton] Fea/warp layout broadcast from upstream triton (#1150)
- [Analysis] Backport layout prerequisites for Triton #11646
Add the input/output basis-mask helpers and update packed transposition selectors to the upstream algorithm preceding #11646. Keep selector ordering correct when low register bits participate in multiple factors, and fold remaining low-bit register permutations into pre-shuffle selectors.
The selector implementation comes from upstream af85fc304db5, the parent of 2074a1b8a5cd (Triton #11646). The basis-mask helpers are backported from the same upstream API with compatibility for FlagTree’s LinearLayout. This commit keeps the existing conversion structure and lowering interfaces.
Validation on H20: NVIDIA/AMD build, pre-commit, five C++ test groups, focused conversion IR tests, and existing warp-local GPU conversion tests.
- [BACKEND] Backport broadcast-local warp shuffles from Triton #11646
Allow layout conversions to read broadcast copies in the current warp or CTA, carrying warp/block-dependent owner lane bits into the shuffle map. Extend mixed transpositions to support one-sided exchanges and retain the existing mixedTranspositions.size() < 2 cost check.
Backport upstream 2074a1b8a5cd283ce584f8ebca825c4f0b215fc1: https://github.com/triton-lang/triton/pull/11646
Keep the required FlagTree compatibility changes with the feature so this commit builds independently: retain tensor-type analysis interfaces, explicit zero padding for the older resizeInDim API, existing register packing/broadcast helpers, and the existing shared-memory fallback. Adapt both common lowering and AMD permlane selection to the new fields. Backport permutation/identity checks as local analysis helpers instead of expanding FlagTree’s public layout APIs. Upstream force_warp_shuffle and GenericLinearEncoding are outside this backport.
Include applicable upstream IR coverage and updated duplication-reduction expectations. Validation: NVIDIA/AMD build, pre-commit, five C++ test groups, and focused NVIDIA/AMD conversion IR tests.
- [Test] Cover broadcast-local layout shuffles in FlagTree
Add the 16x1 pointer layout regression from FlagTree #1047 for NVIDIA and AMD wave64. Check that conversion emits two 32-bit shuffles without shared memory or a barrier. Add layout-unit coverage for local broadcast copies, including partially broadcast warp bits and CTA locality.
Adapt and extend the upstream layout pairs into GPU tests covering forward and reverse conversions, multiple element widths, runtime pointers, owner lanes, packed exchanges, and register selection.
Validation on H20: pre-commit, five C++ test groups, five focused IR files, and 236 GPU cases passed (244 existing cases skipped). All 88 added GPU cases passed. AMD wave64 is validated by compilation, not AMD hardware.
- [AMD][Metax] Preserve legacy layout decomposition in mixed builds
Metax builds the shared AMD permlane lowering against its own pre-#11646 analysis header. Reading regBit and dstLane from that older decomposition breaks compilation in FlagTree #1150.
Use the existing USE_MACA condition to select Metax’s pLane/transposition representation; other builds continue using shuffleMap/regBit/dstLane. Keep the original Metax shuffle gate and analysis implementation unchanged. Annotate each compatibility change with Triton #11646 and FlagTree #1150.
Validation on H20:
- Reproduced the missing-member errors using the actual Metax analysis header; the same AMD translation unit passes syntax checking after the fix.
- NVIDIA/AMD build and pre-commit passed.
- All 24 existing gfx950 permlane cases produce identical LLVM IR before and after the fix.
- Five focused conversion IR files and five C++ test groups passed.
- All 88 broadcast-layout GPU regression cases passed.
The complete Metax SDK build and device tests remain for platform CI.
- [CORE] Isolate layout shuffle backport with the FlagTree feature macro
Define FLAGTREE_SAME_WARP_LAYOUT_SHUFFLE directly in the default NVIDIA/AMD configuration, following the existing FlagTree feature macros.
Wrap all native source differences in the layout backport, including its prerequisites and C++ regression coverage, in matching ifdef/else/endif blocks. Preserve the original declarations, analysis, selectors, and lowering in the disabled branches. Select the legacy AMD interface when the feature is absent, so Metax does not depend on the new decomposition. Keep this feature independent of the TLE macro.
Validation on H20:
- NVIDIA/AMD build, pre-commit, five C++ groups, five IR files, and all 88 broadcast-layout GPU cases passed.
- All 24 gfx950 permlane LLVM IR outputs match the original backport.
- Disabled branches restore every pre-backport nonblank source line.
- Eight CMake backend configurations passed.
- All five affected translation units pass syntax checking with the feature disabled and with it enabled without TLE.
- Shared AMD lowering passes syntax checking with the actual Metax analysis header and the feature disabled.
- [CORE] Match feature guard spacing to the original source
Remove blank lines next to conditional branch boundaries and preserve the original source spacing outside each guard. Drop empty else branches from the three blocks containing only new code.
Validation: pre-commit and git diff –check passed. All 11 feature guards match the original boundary spacing; enabled and disabled code tokens are unchanged across all seven affected native source files.
Co-authored-by: zhzhcookie zhengyang@baai.ac.cn
版权所有:中国计算机学会技术支持:开源发展技术委员会
京ICP备13000930号-9
京公网安备 11010802047560号
FlagTree 是 FlagOS 的一部分。 FlagOS 是一个面向多元AI芯片的开源、统一系统软件栈,旨在打通模型、系统与芯片层,培育开放协作的生态系统。 它支持 “一次开发,多芯运行” 的工作流,兼容多样化的 AI 加速芯片。 它释放硬件性能潜力,消除各类 AI 芯片专用软件栈之间的碎片化问题,并大幅降低大模型在多种 AI 硬件移植与维护的成本。
FlagTree 是面向多种 AI 芯片的开源、统一编译器。 FlagTree 致力于打造多元 AI 芯片编译器及相关工具平台,发展和壮大 Triton 上下游生态。 项目当前处于初期,目标是兼容现有适配方案,统一代码仓库,快速实现单仓库多后端支持。 对于上游模型用户,提供多后端的统一编译能力; 对于下游芯片厂商,提供 Triton 生态接入范例。
多后端支持
各后端基于不同版本的 Triton 适配,因此位于不同的主干分支。 各主干分支均为保护分支且地位相等,表格中所有后端均搭建了 CI/CD Runner。 有些后端适配了多个 Triton 版本,表格中仅展示最新版本。
NVIDIA TileIR
AMD
Enflame(燧原)
ILUVATAR(天数智芯)
HYGON(海光信息)
Moore Threads(摩尔线程)
DAMO ACADEMY(阿里达摩院)
Huixi(辉羲智能)
MetaX(沐曦股份)
Sunrise(曦望芯科)
KLX
T-Head(平头哥)
SpacemiT(进迭时空)
Tsingmicro(清微智能)
tileir
amd
enflame
iluvatar
hcu
mthreads
damoacademy
rpu
metax
sunrise
xpu
ppu
spacemit
tsingmicro
tileir guide
amd guide
enflame guide
iluvatar guide
hcu guide
mthreads guide
-
rpu guide
metax guide
sunrise guide
xpu guide
ppu guide
spacemit guide
tsingmicro guide
ARM64 cpu
x86_64 cpu
cpu
triton-shared
cpu guide
-
FlagTree 的扩展组件当前在部分后端可用:
enflame
TLE-Struct GPU
TLE-Raw
HINTS
mthreads
sunrise
hcu
iluvatar
ppu
TLE-Struct GPU
TLE-Struct DSA
FLIR
FLIR
HINTS
HINTS
TLE(Triton Language Extensions)简介
如果要在 nvidia 后端使用 TLE 语言扩展,请使用 main 分支。 其他后端的 TLE 支持分支详见上面的表格。
Triton 在算子开发效率方面表现突出,但在多元 AI 芯片适配和更深层性能调优场景下,往往需要对分布式执行、内存访问模式和硬件相关原语提供更显式的控制。 TLE 以分层方式扩展 Triton,在保持现有 Triton 工作流兼容性的同时补齐这部分能力。
TLE 的主要优势包括:
Lite/Struct/Raw)。详细设计、API 与示例请参考 TLE Wiki 和 TLE-Raw Wiki。
性能改进
无需修改任何 Triton 算子代码,FlagTree 可在实际模型中的某些形状上获得性能增益。 下面以 Qwen 模型中调用的一些形状下的 mm 算子为例,展示 FlagTree 在不同芯片上的性能增益。
新特性
环境准备
避免环境匹配问题的最佳实践是使用 用户手册 中推荐的镜像。
从源码安装
安装依赖(注意使用正确的 python3.x 执行):
通用的构建安装方式(网络畅通环境下推荐使用):
安装
flagtree后,可通过下列命令查看:免源码安装
参见 用户手册。
关于贡献
欢迎参与 FlagTree 的开发并贡献代码,详情参见 CONTRIBUTING.md。
许可证
FlagTree 使用 MIT license。