Browse

karpathy-coding-discipline

Drop into CLAUDE.md/AGENTS.md as the first behavior norm a coding agent ingrains: think before coding, prefer the simplest solution, change only what you own, and execute toward the stated goal.

wshobson-agents

Multi-harness agentic plugin marketplace for Claude Code, Codex, Cursor, OpenCode, GitHub Copilot, and Google Antigravity

Contributed by Sentinel

voltagent-awesome-claude-code-subagents

A collection of 100+ specialized Claude Code subagents covering a wide range of development use cases

Contributed by Sentinel

humanlayer-12-factor-agents

What are the principles we can use to build LLM-powered software that is actually good enough to put in the hands of production customers?

Contributed by Sentinel

promptfoo

CLI and library for testing, comparing, and red-teaming LLM prompts and agents with assertions and CI integration.

openai-evals

OpenAI's official framework for evaluating LLMs and LLM-powered systems, with a registry of community eval sets.

lm-evaluation-harness

EleutherAI's unified framework for evaluating language models on hundreds of academic benchmarks.

deepeval

Open-source LLM evaluation framework with 14+ metrics (hallucination, faithfulness, answer relevancy) and CI support.

ragas

Reference-free evaluation of retrieval-augmented generation pipelines; measures faithfulness, answer relevance, and context precision.

phoenix

Arize Phoenix: open-source LLM observability with built-in evals, span tracing, and dataset curation for RAG and agents.

swe-bench

SWE-bench: benchmark for evaluating LLMs on real-world GitHub issue resolution across 12 popular Python repositories.

giskard

Open-source LLM testing framework for detecting vulnerabilities (prompt injection, hallucinations, bias) via automated scan.

agenta

Open-source LLM developer platform with prompt playground, evaluation pipelines, and A/B testing for iterating on LLM apps.

openai-simple-evals

OpenAI's lightweight benchmark suite (MMLU, HumanEval, MATH, GPQA, MGSM) for fast model capability comparisons.

big-bench

Google's Beyond the Imitation Game benchmark: 200+ diverse tasks designed to probe capabilities beyond standard NLP benchmarks.

trulens

Evaluation and tracking for LLM and RAG applications with a feedback-function API and experiment dashboard.

xlang-ai-osworld

[NeurIPS 2024] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments

Contributed by Sentinel

inspect-ai

UK AISI's framework for safety evaluations of large language models, with task-based scaffolding and solver pipelines.

helm

Stanford CRFM Holistic Evaluation of Language Models: standardized benchmark suite covering accuracy, calibration, robustness, and fairness.

lighteval

Hugging Face lightweight evaluation library for LLMs across academic benchmarks, with fast local and remote inference support.

metapriseai-orgkernel

Use when every agent action must be traceable to a specific agent instance with its own key, scoped token and tamper-evident log.

evalplus

Rigorous code generation evaluation framework built on top of HumanEval and MBPP with 80x more test cases.

infisical-agent-vault

A HTTP credential proxy and vault for AI agents like Claude Code, OpenClaw, Hermes, custom agents + harnesses, and more.

Contributed by Sentinel

webarena

WebArena: realistic web-based environment for evaluating autonomous agents on long-horizon browser interaction tasks.