Browse

wandb-weave-evals

Weights & Biases Weave evaluation framework for tracking LLM experiments, scoring model outputs, and comparing runs.

langtrace

Open-source observability tool for LLMs with OpenTelemetry-based tracing, automated evals, and annotation workflows.

letta-ai-agent-file

Use when you need to save, share, version or move a whole agent — its persona, memory and behaviour — as one portable file.

xiaowu0162-longmemeval

Benchmarking Chat Assistants on Long-Term Interactive Memory (ICLR 2025)

Contributed by Sentinel

princeton-nlp-webshop

[NeurIPS 2022] 🛒WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents

Contributed by Sentinel

unicity-aos-capsule-identity

Use when an agent's identity must be persisted as state and assembled into its system prompt at boot, rather than pasted into a prompt by hand.

stonybrooknlp-appworld

🌍 AppWorld: A Controllable World of Apps and People for Benchmarking Function Calling and Interactive Coding Agent, ACL'24 Best Resource Paper.

Contributed by Sentinel

continuous-eval

Relari's modular evaluation library for LLM pipelines with deterministic + LLM-based metrics for RAG and agent workflows.

agntcy-oasf

Use when agents must describe themselves to other systems in a common schema so they can be catalogued, discovered and verified.

telagod-code-abyss

Use when you want a coding agent to have a consistent, composable personality and voice across Claude Code, Codex, Gemini CLI and OpenClaw.

agntcy-dir

Use when agents and multi-agent systems need to announce themselves and be found across organisations rather than hardcoded.

metr-task-standard

METR's Task Standard: a specification and scaffold for creating agentic tasks used in autonomous agent capability evaluations.

harbor-framework-terminal-bench-2-1

Terminal-Bench 2.1

Contributed by Sentinel

character-card-spec-v2

Use when reading or writing the character-card files that the roleplay-agent ecosystem actually ships, including the PNG-embedded variant.

cloudflare-web-bot-auth

Use when a website has to be able to tell that a request really came from your agent and not from someone impersonating it.

highflame-ai-zeroid

Use when a fleet of autonomous agents needs issued identities with a lifecycle — created, rotated and revoked.

agentmail-to-agentmail-toolkit

Use when an agent needs its own email address so people and systems can reach it, and it can act on what arrives.

agntcy-identity

Use when agents, MCP servers and multi-agent systems all need issued identities that another party can verify.

vellum-evals

Vellum evaluation SDK for running LLM test suites with custom metrics, dataset pinning, and CI workflow integration.

character-card-spec-v3

Use when authoring an agent persona against the current character-card standard, with lorebooks, assets and decorators.

agentrhq-authsome

Use when agents must stay logged in to third-party services without ever seeing your credentials.

gebruder-wirken

Use when autonomous agents need one gateway that holds their credentials, isolates them per channel, and logs every session.

opena2a-agent-identity-management

Use when non-human identities need the same lifecycle a workforce IAM gives people — issue, authorize, audit, revoke.

vestauth

Use when an agent needs to authenticate as itself to services, without you hand-rolling keys and rotation.