AutoRed
Ask harder questions of AI.
Tech Stack
About
AutoRed is a research exploration into automated red-teaming of large language models. Rather than relying on human adversaries to probe AI safety, AutoRed investigates algorithmic approaches — using tree search strategies such as MCTS and A* to generate adversarial prompts systematically. The goal is to produce deterministic, reproducible evaluation results rather than one-off findings.
The Problem
LLM safety evaluations are often ad-hoc, human-driven, and difficult to reproduce. There is no standard pipeline for algorithmically generating and scoring adversarial prompts at scale.
The Approach
AutoRed explores adversarial prompt search using Monte Carlo Tree Search (MCTS) and A*, paired with deterministic evaluation oracles that score model outputs against safety criteria.
Key Features
- Adversarial prompt generation via MCTS and A*
- Deterministic evaluation oracles for reproducibility
- FastAPI-based evaluation server
- Structured findings format for comparison across runs
Highlights
- Explored adversarial prompt search using MCTS and A* strategies.
- Outlined deterministic evaluation oracles and reproducible findings.