Browse

big-bench

Google's Beyond the Imitation Game benchmark: 200+ diverse tasks designed to probe capabilities beyond standard NLP benchmarks.

trulens

Evaluation and tracking for LLM and RAG applications with a feedback-function API and experiment dashboard.

claude-devtools

A desktop app that shows your Claude Code sessions by reading their logs: context use per turn across categories, compaction, sub-agent execution trees and custom notification triggers.

xlang-ai-osworld

[NeurIPS 2024] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments

Contributed by Sentinel

inspect-ai

UK AISI's framework for safety evaluations of large language models, with task-based scaffolding and solver pipelines.

letta-ai-letta-code

Stateful agents that are like people, with memory, identity, and the ability to learn and adapt

Contributed by Sentinel

helm

Stanford CRFM Holistic Evaluation of Language Models: standardized benchmark suite covering accuracy, calibration, robustness, and fairness.

lighteval

Hugging Face lightweight evaluation library for LLMs across academic benchmarks, with fast local and remote inference support.

metapriseai-orgkernel

Use when every agent action must be traceable to a specific agent instance with its own key, scoped token and tamper-evident log.

crystal

A full-fledged desktop application for orchestrating, monitoring, and interacting with Claude Code agents.

omnara

A command center for AI agents that syncs Claude Code sessions across terminal, web, and mobile. Allows for remote monitoring, human-in-the-loop interaction, and team collaboration.

primeintellect-ai-prime-rl

Agentic RL Training at Scale

Contributed by Sentinel

tweakcc

Command-line tool to customize your Claude Code styling.

evalplus

Rigorous code generation evaluation framework built on top of HumanEval and MBPP with 80x more test cases.

infisical-agent-vault

A HTTP credential proxy and vault for AI agents like Claude Code, OpenClaw, Hermes, custom agents + harnesses, and more.

Contributed by Sentinel

webarena

WebArena: realistic web-based environment for evaluating autonomous agents on long-horizon browser interaction tasks.

claude-code-tools

Well-crafted toolset for session continuity, featuring skills/commands to avoid compaction and recover context across sessions with cross-agent handoff between Claude Code and Codex CLI. Includes a fast Rust/Tantivy-powered full-text session search (TUI for humans, skill/CLI for agents), tmux-cli skill + command for interacting with scripts and CLI agents, and safety hooks to block dangerous commands.

cc-sessions

An opinionated approach to productive development with Claude Code

tau-bench

Tau-bench: agent benchmark for tool-agent-user interactions in retail and airline domains with policy-grounded evaluation.

harbor-framework-terminal-bench

Measuring and evolving with the frontier of agent work

Contributed by Sentinel

claude-code-ide-el

claude-code-ide.el integrates Claude Code with Emacs, like Anthropic’s VS Code/IntelliJ extensions. It shows ediff-based code suggestions, pulls LSP/flymake/flycheck diagnostics, and tracks buffer context. It adds an extensible MCP tool support for symbol refs/defs, project metadata, and tree-sitter AST queries.

rulesync

A Node.js CLI tool that automatically generates configs (rules, ignore files, MCP servers, commands, and subagents) for various AI coding agents. Rulesync can convert configs between Claude Code and other AI agents in both directions.

claude-code-nvim

A seamless integration between Claude Code AI assistant and Neovim.

wandb-weave-evals

Weights & Biases Weave evaluation framework for tracking LLM experiments, scoring model outputs, and comparing runs.