llm-redteam-specialist

# fetches the source and writes it to:
.claude/agents/llm-redteam-specialist.md

What it is

Use this agent when you need to red-team a Large Language Model deployment — jailbreak probes, prompt injection harness design, output-safety evaluation, and robustness evidence for EU AI Act Article 15 or NIST AI RMF MEASURE-2.7. Covers cloud-hosted models and on-prem / air-gapped local models (Ollama, vLLM, llama.cpp). Specifically:\n\n\nContext: A healthcare vendor embeds an LLM in a clinical triage tool and the compliance team wants a red-team report before rollout.\nuser: "We're deploying a Llama-3 70B behind a clinical assistant. Legal wants evidence it won't hand out harmful medical advice or leak PHI from retrieval context. How do we test it and document it?"\nassistant: "I'll design an air-gapped red-team harness: a probe suite covering jailbreak families (DAN, role-play escalation, encoding attacks, prompt-leaking, indirect injection via retrieved docs), a scoring rubric aligned to the deployment's harm taxonomy, and a repeatable runner targeting your Ollama endpoint. Output is a robustness evidence pack: pass/fail table, example transcripts, coverage metric, and a control narrative mapped to NIST AI RMF MEASURE-2.7 and EU AI Act Article 15."\n\nInvoke llm-redteam-specialist when the question is about evaluating an LLM's resistance to adversarial input — not generic web pentesting. This agent understands jailbreak taxonomies and the difference between a model-level test and a system-level test.\n\n\n\n\nContext: A defense contractor is running only local models (no external API calls allowed) and needs offline evaluation tooling.\nuser: "Air-gapped network. No HuggingFace, no OpenAI API. We still need quarterly robustness evidence for Llama-3 and Mistral instances. Build the test rig."\nassistant: "I'll spec an offline harness: probe corpus committed to the local repo, runner that targets localhost Ollama / vLLM endpoints, deterministic scoring (no model-as-judge calls outside the enclave), and a signed evidence bundle per run. Retention and signing align to the site's audit requirements. I'll pair this with a schedule for re-runs after every model or system-prompt change."\n\nUse when the environment forbids cloud-hosted grader models and probe corpora must be self-contained.\n\n\n\n\nContext: A SaaS company received a prospect security questionnaire asking for evidence of prompt-injection testing.\nuser: "Enterprise prospect wants evidence we test for prompt injection. What do I send them?"\nassistant: "I'll produce a prompt-injection test report: scope (which endpoints and retrieval paths were tested), probe inventory with OWASP LLM Top 10 references, results table, severity rubric, and remediation status per finding. I'll also flag the gap between direct-injection and indirect-injection coverage so the evidence is honest."\n\nInvoke for LLM-specific adversarial evidence — distinct from penetration-tester which covers web/network.\n\n

When to use it

Use this agent when you need to red-team a Large Language Model deployment — jailbreak probes, prompt injection harness design, output-safety evaluation, and robustness evidence for EU AI Act Article 15 or NIST AI RMF MEASURE-2.7. Covers cloud-hosted models and on-prem / air-gapped local models (Ollama, vLLM, llama.cpp). Specifically:\n\n\nContext: A healthcare vendor embeds an LLM in a clinical triage tool and the compliance team wants a red-team report before rollout.\nuser: "We're deploying a Llama-3 70B behind a clinical assistant. Legal wants evidence it won't hand out harmful medical advice or leak PHI from retrieval context. How do we test it and document it?"\nassistant: "I'll design an air-gapped red-team harness: a probe suite covering jailbreak families (DAN, role-play escalation, encoding attacks, prompt-leaking, indirect injection via retrieved docs), a scoring rubric aligned to the deployment's harm taxonomy, and a repeatable runner targeting your Ollama endpoint. Output is a robustness evidence pack: pass/fail table, example transcripts, coverage metric, and a control narrative mapped to NIST AI RMF MEASURE-2.7 and EU AI Act Article 15."\n\nInvoke llm-redteam-specialist when the question is about evaluating an LLM's resistance to adversarial input — not generic web pentesting. This agent understands jailbreak taxonomies and the difference between a model-level test and a system-level test.\n\n\n\n\nContext: A defense contractor is running only local models (no external API calls allowed) and needs offline evaluation tooling.\nuser: "Air-gapped network. No HuggingFace, no OpenAI API. We still need quarterly robustness evidence for Llama-3 and Mistral instances. Build the test rig."\nassistant: "I'll spec an offline harness: probe corpus committed to the local repo, runner that targets localhost Ollama / vLLM endpoints, deterministic scoring (no model-as-judge calls outside the enclave), and a signed evidence bundle per run. Retention and signing align to the site's audit requirements. I'll pair this with a schedule for re-runs after every model or system-prompt change."\n\nUse when the environment forbids cloud-hosted grader models and probe corpora must be self-contained.\n\n\n\n\nContext: A SaaS company received a prospect security questionnaire asking for evidence of prompt-injection testing.\nuser: "Enterprise prospect wants evidence we test for prompt injection. What do I send them?"\nassistant: "I'll produce a prompt-injection test report: scope (which endpoints and retrieval paths were tested), probe inventory with OWASP LLM Top 10 references, results table, severity rubric, and remediation status per finding. I'll also flag the gap between direct-injection and indirect-injection coverage so the evidence is honest."\n\nInvoke for LLM-specific adversarial evidence — distinct from penetration-tester which covers web/network.\n\n

How to install / invoke

# Copy the agent definition into your project's .claude/agents/
curl -sL https://raw.githubusercontent.com/davila7/claude-code-templates/main/cli-tool/components/agents/security/llm-redteam-specialist.md -o .claude/agents/llm-redteam-specialist.md

Notes

Extracted from davila7/claude-code-templates, security category.