AI safety research·AI / ML·2026

AutoRed

Ask harder questions of AI.

Concept / Exploration

Tech Stack

PythonFastAPILLM Evaluation

About

AutoRed is a research exploration into automated red-teaming of large language models. Rather than relying on human adversaries to probe AI safety, AutoRed investigates algorithmic approaches — using tree search strategies such as MCTS and A* to generate adversarial prompts systematically. The goal is to produce deterministic, reproducible evaluation results rather than one-off findings.

The Problem

LLM safety evaluations are often ad-hoc, human-driven, and difficult to reproduce. There is no standard pipeline for algorithmically generating and scoring adversarial prompts at scale.

The Approach

AutoRed explores adversarial prompt search using Monte Carlo Tree Search (MCTS) and A*, paired with deterministic evaluation oracles that score model outputs against safety criteria.

Key Features

  • Adversarial prompt generation via MCTS and A*
  • Deterministic evaluation oracles for reproducibility
  • FastAPI-based evaluation server
  • Structured findings format for comparison across runs

Highlights

  • Explored adversarial prompt search using MCTS and A* strategies.
  • Outlined deterministic evaluation oracles and reproducible findings.