[CICD] Support Agent asistant in Workflow (#1192)
PR Category
CICD
PR Types
New Features 
PR Description
Add AI-Powered CI Failure Analysis and Interactive Code Assistant
Workflows
Summary
Adds two Claude AI-powered GitHub Actions workflows to automate test
failure analysis and enable interactive code collaboration via @claude
mentions.
Changes
test_failure_analysis.yml — Automated Test Failure Analysis
Triggered when any of the following workflows fail: ascend_tests,
cuda_tests, metax_c500_tests, format_check.
What it does:

Flaky test detection — Analyzes CI logs to determine if a failure
is intermittent (timeouts, race conditions, network errors, etc.)

High confidence (≥0.7): automatically reruns the failed workflow
Low confidence: logs a warning, recommends manual review


Deep failure analysis (PR only) — When the failure is a real bug
on a PR:

Fetches failed job logs and PR diff




Performs root cause analysis correlating code changes with test
failures
Generates actionable fix suggestions with file paths and code snippets

Posts a structured analysis comment on the PR



Safety:

Runs in read-only mode — Claude cannot modify files, push commits, or
create branches
Tool whitelist: only Bash(gh *), Read, Glob, Grep are allowed
Explicit read-only instruction in the prompt as an additional
safeguard

Configuration:

Model controlled via CLAUDE_MODEL repository variable
API endpoint via ANTHROPIC_BASE_URL secret

claude.yml — Interactive AI Code Assistant
Triggered when @claude is mentioned in issue comments, PR review
comments, PR reviews, or issue titles/bodies.

Responds to user requests for code review, debugging, code changes,
etc.
Uses mco-4 model with write permissions for code modification
scenarios

Technical Details

Structured output: Uses --json-schema to enforce standardized
JSON responses from Claude, enabling reliable downstream parsing
Shell-safe JSON handling: Writes Claude output to temp files via
heredoc, then reads with jq — avoids single-quote breakage when JSON
contains apostrophes (e.g. what's, can't)
Stable PR identification: Uses
github.event.workflow_run.pull_requests[0].number directly from the
event payload instead of gh pr list --head which is unreliable for
fork PRs
Debug step: Includes a Debug detect output step with if: always() to surface Claude’s raw response even when subsequent steps
fail

PR Comment Example
When a real bug is detected, the workflow posts:
## ❌ Test Failure Analysis

🔴 **Severity**: critical

### Failed Tests
- `test_distributed_optimizer`

### Root Cause
The learning rate scheduler change causes NaN gradients in distributed training with mixed precision.

### Suggested Fixes
### 1. Set minimum learning rate to avoid zero division
📄 **File**: `flagscale/train/optimizer.py`

---
**Flaky Detection**: Not a flaky test (confidence: 0.9)
Permissions

test_failure_analysis.yml: contents: read, actions: write,
pull-requests: write, id-token: write
claude.yml: contents: write, pull-requests: write, issues: write, id-token: write, actions: read